[8592364.070945] SysRq : Show State [Tue Mar 17 17:57:59 2020][8592364.074216] task PC stack pid father [Tue Mar 17 17:57:59 2020][8592364.080349] systemd S ffff9c7ae9ed0000 0 1 0 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.087658] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.090292] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.095478] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:57:59 2020][8592364.102708] [] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0 [Tue Mar 17 17:57:59 2020][8592364.109715] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:57:59 2020][8592364.116259] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:57:59 2020][8592364.121490] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:57:59 2020][8592364.127281] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:57:59 2020][8592364.133064] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:57:59 2020][8592364.139285] kthreadd S ffff9c7ae9ed1040 0 2 0 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.146573] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.149222] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.154371] [] kthreadd+0x2f5/0x300 [Tue Mar 17 17:57:59 2020][8592364.159706] [] ? kthread_create_on_cpu+0x60/0x60 [Tue Mar 17 17:57:59 2020][8592364.166158] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:57:59 2020][8592364.172802] [] ? kthread_create_on_cpu+0x60/0x60 [Tue Mar 17 17:57:59 2020][8592364.179275] ksoftirqd/0 S ffff9c4b69b18000 0 3 2 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.186566] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.189224] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.194392] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:57:59 2020][8592364.200423] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.206455] [] kthread+0xd1/0xe0 [Tue Mar 17 17:57:59 2020][8592364.211534] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.217827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:57:59 2020][8592364.224470] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.230767] kworker/0:0H S ffff9c4b69b1a080 0 5 2 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.238085] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.240733] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:57:59 2020][8592364.246400] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:57:59 2020][8592364.253139] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.258284] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:57:59 2020][8592364.264054] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:57:59 2020][8592364.270781] [] kthread+0xd1/0xe0 [Tue Mar 17 17:57:59 2020][8592364.275857] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.282148] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:57:59 2020][8592364.288780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.295075] migration/0 S ffff9c4b69b1d140 0 8 2 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.302376] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.305017] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.310178] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:57:59 2020][8592364.316208] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.322231] [] kthread+0xd1/0xe0 [Tue Mar 17 17:57:59 2020][8592364.327300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.333591] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:57:59 2020][8592364.340222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.346508] rcu_bh S ffff9c4b69b1e180 0 9 2 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.353795] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.356425] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:57:59 2020][8592364.362124] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.367268] [] rcu_gp_kthread+0xb2/0x710 [Tue Mar 17 17:57:59 2020][8592364.373039] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:57:59 2020][8592364.379069] [] ? rcu_gp_fqs+0x90/0x90 [Tue Mar 17 17:57:59 2020][8592364.384587] [] kthread+0xd1/0xe0 [Tue Mar 17 17:57:59 2020][8592364.389656] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.395945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:57:59 2020][8592364.402580] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:57:59 2020][8592364.408865] rcu_sched S ffff9c4b69b80000 0 10 2 0x00000000 [Tue Mar 17 17:57:59 2020][8592364.416153] Call Trace: [Tue Mar 17 17:57:59 2020][8592364.418798] [] schedule+0x29/0x70 [Tue Mar 17 17:57:59 2020][8592364.423945] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:00 2020][8592364.429978] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:00 2020][8592364.436516] [] ? prepare_to_wait+0x56/0x90 [Tue Mar 17 17:58:00 2020][8592364.442463] [] rcu_gp_kthread+0x349/0x710 [Tue Mar 17 17:58:00 2020][8592364.448320] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:00 2020][8592364.454336] [] ? rcu_gp_fqs+0x90/0x90 [Tue Mar 17 17:58:00 2020][8592364.459845] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.464910] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.471196] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.477830] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.484104] lru-add-drain S ffff9c4b69b81040 0 11 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.491403] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.494049] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.499208] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:00 2020][8592364.505064] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:00 2020][8592364.511263] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.516338] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.522624] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.529253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.535539] watchdog/0 S ffff9c4b69b82080 0 12 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.542816] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.545462] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:00 2020][8592364.550959] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.556134] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.562145] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.568152] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.573228] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.579515] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.586136] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.592441] watchdog/1 S ffff9c5ae9432080 0 13 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.599715] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.602343] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:00 2020][8592364.607852] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.613009] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.619022] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.625049] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.630134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.636406] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.643057] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.649333] migration/1 S ffff9c5ae94330c0 0 14 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.656627] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.659258] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.664422] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.670455] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.676488] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.681564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.687840] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.694474] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.700781] ksoftirqd/1 S ffff9c5ae9434100 0 15 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.708055] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.710732] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.715875] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.721912] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.727924] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.736532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.742858] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.749478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.755766] kworker/1:0H S ffff9c5ae9459040 0 17 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.763067] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.765703] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:00 2020][8592364.771890] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.777055] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:00 2020][8592364.782809] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:00 2020][8592364.789533] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.794601] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.800888] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.807524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.813808] watchdog/2 S ffff9c6ae98c2080 0 19 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.821095] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.823742] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:00 2020][8592364.829933] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:00 2020][8592364.835444] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.840590] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.846647] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.852671] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.857739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.864011] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.870678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.876962] migration/2 S ffff9c6ae98c30c0 0 20 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.884259] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.886904] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.892063] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.898093] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.904135] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.909213] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.915509] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.922145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.928437] ksoftirqd/2 S ffff9c6ae98c4100 0 21 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.935731] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.938360] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592364.943522] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592364.949545] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.955569] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592364.960639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.966925] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592364.973563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592364.979845] kworker/2:0H S ffff9c6ae98e9040 0 23 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592364.987141] Call Trace: [Tue Mar 17 17:58:00 2020][8592364.989791] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:00 2020][8592364.995975] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.001138] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:00 2020][8592365.006887] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:00 2020][8592365.013635] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.018692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.024993] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.031630] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.037912] watchdog/3 S ffff9c7ae9fab0c0 0 25 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.045202] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.047833] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:00 2020][8592365.054018] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:00 2020][8592365.059542] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.064685] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.070716] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.076737] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.081808] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.088093] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.094721] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.101004] migration/3 S ffff9c7ae9fd0000 0 26 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.108301] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.110945] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.116105] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.122142] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.128167] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.133237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.139525] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.146159] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.152443] ksoftirqd/3 S ffff9c7ae9fd1040 0 27 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.159731] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.162359] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.167526] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.173553] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.179581] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.184641] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.190933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.197561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.203847] kworker/3:0H S ffff9c7ae9fd30c0 0 29 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.211148] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.213796] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:00 2020][8592365.219992] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.225133] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:00 2020][8592365.230887] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:00 2020][8592365.237624] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.242684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.248978] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.255610] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.261898] watchdog/4 S ffff9c4b6944b0c0 0 31 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.269193] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.271838] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:00 2020][8592365.278027] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:00 2020][8592365.283552] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.288695] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.294725] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.300745] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.305819] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.312101] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.318732] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.325014] migration/4 S ffff9c4b69498000 0 32 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.332311] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.334958] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:00 2020][8592365.341261] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.346432] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.352479] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.358501] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.363579] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.369864] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.376495] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.382778] ksoftirqd/4 S ffff9c4b69499040 0 33 2 0x00000000 [Tue Mar 17 17:58:00 2020][8592365.390065] Call Trace: [Tue Mar 17 17:58:00 2020][8592365.392710] [] schedule+0x29/0x70 [Tue Mar 17 17:58:00 2020][8592365.397869] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:00 2020][8592365.403898] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.409924] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:00 2020][8592365.414990] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.421292] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:00 2020][8592365.427960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:00 2020][8592365.434270] kworker/4:0H S ffff9c4b6949b0c0 0 35 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.441576] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.444229] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:01 2020][8592365.450417] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.455600] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:01 2020][8592365.461355] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:01 2020][8592365.468079] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.473154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.479442] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.486076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.492360] watchdog/5 S ffff9c5ae945e180 0 36 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.499648] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.502293] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:01 2020][8592365.508476] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:01 2020][8592365.513999] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.519144] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.525179] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.531191] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.536266] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.542552] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.549189] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.555472] migration/5 S ffff9c5ae945d140 0 37 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.562758] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.565393] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.570552] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.576577] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.582627] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.587689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.593980] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.600615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.606903] ksoftirqd/5 S ffff9c5ae945c100 0 38 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.614206] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.616853] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.622001] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.628044] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.634053] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.639145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.645418] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.652054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.658336] kworker/5:0H S ffff9c5ae9498000 0 40 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.665630] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.668279] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:01 2020][8592365.674476] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.679639] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:01 2020][8592365.685404] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:01 2020][8592365.692119] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.697193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.703480] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.710112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.716393] watchdog/6 S ffff9c6ae98ee180 0 41 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.723684] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.726327] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:01 2020][8592365.731831] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.736986] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.743011] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.749017] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.754087] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.760370] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.767001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.773282] migration/6 R running task 0 42 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.780573] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.783229] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.788391] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.794413] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.800422] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.805514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.811802] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.818436] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.824720] ksoftirqd/6 R running task 0 43 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.832009] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.834655] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.839803] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.845832] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.851841] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.856922] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.863197] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.869832] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.876098] kworker/6:0H S ffff9c6ae9928000 0 45 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.883388] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.886024] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:01 2020][8592365.892229] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.897391] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:01 2020][8592365.903158] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:01 2020][8592365.909879] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.914952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.921242] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.927874] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.934161] watchdog/7 S ffff9c7ae9fd6180 0 46 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.941452] Call Trace: [Tue Mar 17 17:58:01 2020][8592365.944096] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:01 2020][8592365.949603] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592365.954764] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592365.960787] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.966810] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592365.971905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.978191] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592365.984827] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592365.991112] migration/7 S ffff9c7ae9fd5140 0 47 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592365.998401] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.001044] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.006186] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.012206] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.018231] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.023301] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.029593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.036222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.042508] ksoftirqd/7 S ffff9c7ae9818000 0 48 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.049815] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.052454] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.057600] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.063627] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.069641] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.074713] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.080982] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.087618] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.093902] kworker/7:0H S ffff9c7ae981a080 0 50 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.101204] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.103852] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:01 2020][8592366.110042] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.115216] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:01 2020][8592366.120970] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:01 2020][8592366.127695] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.132770] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.139055] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.145683] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.151968] watchdog/8 S ffff9c4b6949e180 0 51 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.159237] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.161883] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:01 2020][8592366.167371] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.172549] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.178558] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.184600] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.189660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.195945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.202585] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.208874] migration/8 S ffff9c4b6949d140 0 52 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.216143] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.218791] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.223938] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.229971] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.235986] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.241075] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.247348] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.253998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.260269] ksoftirqd/8 S ffff9c4b6949c100 0 53 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.267590] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.270219] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.275381] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.281402] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.287429] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.292496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.298784] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.305417] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.311732] kworker/8:0H S ffff9c4b694e9040 0 55 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.319023] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.321676] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:01 2020][8592366.327860] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.333032] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:01 2020][8592366.338783] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:01 2020][8592366.345505] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.350575] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.356850] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:01 2020][8592366.363478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.369747] watchdog/9 S ffff9c5ae949e180 0 56 2 0x00000000 [Tue Mar 17 17:58:01 2020][8592366.377059] Call Trace: [Tue Mar 17 17:58:01 2020][8592366.379688] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:01 2020][8592366.385184] [] schedule+0x29/0x70 [Tue Mar 17 17:58:01 2020][8592366.390347] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:01 2020][8592366.396354] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.402399] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:01 2020][8592366.407456] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:01 2020][8592366.413747] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.420377] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.426661] migration/9 S ffff9c5ae949d140 0 57 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.433946] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.436578] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:02 2020][8592366.442867] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.448037] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.454047] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.460067] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.465123] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.471406] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.478034] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.484322] ksoftirqd/9 S ffff9c5ae949c100 0 58 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.491616] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.494263] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.499411] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.505436] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.511450] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.516538] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.522810] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.529446] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.535726] kworker/9:0H S ffff9c5ae949a080 0 60 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.543023] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.545670] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:02 2020][8592366.551867] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.557022] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:02 2020][8592366.562788] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:02 2020][8592366.569501] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.574560] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.580849] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.587481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.593762] watchdog/10 S ffff9c6ae992e180 0 61 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.601033] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.603677] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:02 2020][8592366.609181] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.614338] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.620360] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.626381] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.631456] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.637735] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.644368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.650649] migration/10 S ffff9c6ae992d140 0 62 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.657940] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.660569] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.665747] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.671758] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.677786] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.682842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.689168] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.695782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.702094] ksoftirqd/10 S ffff9c6ae992c100 0 63 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.709367] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.712014] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.717160] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.723215] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.729231] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.734307] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.740600] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.747237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.753527] kworker/10:0H S ffff9c6ae992a080 0 65 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.760827] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.763459] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:02 2020][8592366.769157] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:02 2020][8592366.775862] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.781010] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:02 2020][8592366.786823] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:02 2020][8592366.793552] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.798626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.804911] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.811537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.817826] watchdog/11 S ffff9c7ae981e180 0 66 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.825120] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.827765] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:02 2020][8592366.833254] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.838443] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.844450] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.850493] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.855551] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.861847] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.868467] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.874769] migration/11 S ffff9c7ae981d140 0 67 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.882043] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.884704] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:02 2020][8592366.890975] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.896151] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.902163] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.908193] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.913249] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.919551] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.926170] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.932459] ksoftirqd/11 S ffff9c7ae981c100 0 68 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.939748] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.942393] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592366.947543] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592366.953594] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.959604] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592366.964694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.970973] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592366.977606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592366.983895] kworker/11:0H S ffff9c7ae9848000 0 70 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592366.991192] Call Trace: [Tue Mar 17 17:58:02 2020][8592366.993834] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:02 2020][8592367.000035] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.005195] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:02 2020][8592367.010961] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:02 2020][8592367.017690] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.022773] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.029058] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.035709] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.041996] watchdog/12 S ffff9c4b694ee180 0 71 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.049302] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.051946] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:02 2020][8592367.057456] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.062615] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.068638] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.074664] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.079734] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.086021] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.092655] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.098935] migration/12 S ffff9c4b694ed140 0 72 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.106225] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.108868] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.114027] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.120049] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.126075] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.131148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.137437] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.144065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.150349] ksoftirqd/12 S ffff9c4b694ec100 0 73 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.157636] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.160280] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.165425] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.171465] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.177478] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.182565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.188839] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.195488] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.201761] kworker/12:0H S ffff9c4b694ea080 0 75 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.209053] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.211684] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:02 2020][8592367.218615] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.223771] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:02 2020][8592367.229549] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:02 2020][8592367.236269] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.241362] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.247665] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.254299] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.260595] watchdog/13 S ffff9c5ae9499040 0 76 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.267884] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.270515] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:02 2020][8592367.276027] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.281189] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.287216] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.293251] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.298326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.304601] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.311242] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.317527] migration/13 S ffff9c5ae94d8000 0 77 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.324798] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.327434] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:02 2020][8592367.333718] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.338860] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.344866] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.350885] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.355989] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.362259] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:02 2020][8592367.368893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.375178] ksoftirqd/13 S ffff9c5ae94d9040 0 78 2 0x00000000 [Tue Mar 17 17:58:02 2020][8592367.382450] Call Trace: [Tue Mar 17 17:58:02 2020][8592367.385095] [] schedule+0x29/0x70 [Tue Mar 17 17:58:02 2020][8592367.390255] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:02 2020][8592367.396282] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.402311] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:02 2020][8592367.407384] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:02 2020][8592367.413674] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.420316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.426637] kworker/13:0H S ffff9c5ae94db0c0 0 80 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.433936] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.436570] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:03 2020][8592367.442755] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.447925] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:03 2020][8592367.453680] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:03 2020][8592367.460417] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.465474] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.471763] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.478378] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.484683] watchdog/14 S ffff9c6ae9929040 0 81 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.491954] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.494618] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:03 2020][8592367.500800] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:03 2020][8592367.506321] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.511467] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.517501] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.523516] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.528626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.534895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.541529] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.547797] migration/14 S ffff9c8a3a0fd140 0 82 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.555084] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.557729] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:03 2020][8592367.564014] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.569176] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.575201] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.581219] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.586278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.592570] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.599210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.605494] ksoftirqd/14 S ffff9c6ae9979040 0 83 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.612771] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.615398] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.620561] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.626574] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.632612] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.637685] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.644008] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.650642] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.656936] kworker/14:0H S ffff9c6ae997b0c0 0 85 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.664231] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.666862] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:03 2020][8592367.672528] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:03 2020][8592367.679242] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.684384] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:03 2020][8592367.690144] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:03 2020][8592367.696866] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.701954] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.708228] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.714861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.721151] watchdog/15 S ffff9c7ae984e180 0 86 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.728446] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.731092] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:03 2020][8592367.736589] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.741746] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.747756] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.753787] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.758869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.765159] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.771786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.778069] migration/15 S ffff9c7ae984d140 0 87 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.785341] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.787990] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:03 2020][8592367.794262] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.799440] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.805451] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.811480] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.816549] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.822819] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.829454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.835745] ksoftirqd/15 S ffff9c7ae984c100 0 88 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.843036] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.845681] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.850821] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.858610] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.864619] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.869757] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.876038] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.882671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.888952] kworker/15:0H S ffff9c7ae984a080 0 90 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.896245] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.898893] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:03 2020][8592367.905090] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.910248] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:03 2020][8592367.916014] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:03 2020][8592367.922736] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.927806] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.934100] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592367.940744] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.947025] watchdog/16 S ffff9c4b69578000 0 91 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592367.954318] Call Trace: [Tue Mar 17 17:58:03 2020][8592367.956960] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:03 2020][8592367.963161] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:03 2020][8592367.968670] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592367.973827] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592367.979850] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.985875] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592367.990952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592367.997238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.003871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.010160] migration/16 S ffff9c4b69579040 0 92 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.017455] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.020099] [] ? cpu_stop_unpark+0x50/0x50 [Tue Mar 17 17:58:03 2020][8592368.026018] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.031187] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592368.037201] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.043245] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.048301] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.054626] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.061240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.067534] ksoftirqd/16 S ffff9c4b6957a080 0 93 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.074809] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.077455] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.082594] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592368.088631] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.094640] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.099694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.105990] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.112624] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.118890] kworker/16:0H S ffff9c4b6957c100 0 95 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.126172] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.128807] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:03 2020][8592368.135011] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.140166] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:03 2020][8592368.145930] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:03 2020][8592368.152646] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.157717] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.164020] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.170681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.176962] watchdog/17 S ffff9c5ae94de180 0 96 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.184263] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.186906] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:03 2020][8592368.192408] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.197564] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592368.203574] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.209579] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.214655] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.220952] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.227591] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.233879] migration/17 S ffff9c5ae94dd140 0 97 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.241168] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.243808] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:03 2020][8592368.250097] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.255249] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592368.261277] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.267297] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.272368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.278664] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.285299] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.291578] ksoftirqd/17 S ffff9c5ae94dc100 0 98 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.298871] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.301498] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.306666] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:03 2020][8592368.312678] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.318702] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.323770] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.330055] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.336696] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.342986] kworker/17:0H S ffff9c5ae9529040 0 100 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.350277] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.352926] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:03 2020][8592368.359106] [] schedule+0x29/0x70 [Tue Mar 17 17:58:03 2020][8592368.367338] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:03 2020][8592368.373090] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:03 2020][8592368.379853] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:03 2020][8592368.384909] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.391218] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:03 2020][8592368.397841] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:03 2020][8592368.404161] watchdog/18 S ffff9c6ae997e180 0 101 2 0x00000000 [Tue Mar 17 17:58:03 2020][8592368.411435] Call Trace: [Tue Mar 17 17:58:03 2020][8592368.414097] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:03 2020][8592368.420281] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:03 2020][8592368.425808] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.430958] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.436988] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.443021] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.448082] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.454390] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.461012] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.467316] migration/18 S ffff9c6ae997d140 0 102 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.474590] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.477233] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:04 2020][8592368.483502] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.488677] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.494690] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.500738] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.505795] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.512089] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.518723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.525018] ksoftirqd/18 S ffff9c6ae997c100 0 103 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.532309] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.534938] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.540089] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.546133] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.552140] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.557222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.563493] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.570147] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.576416] kworker/18:0H S ffff9c6ae99d1040 0 105 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.583722] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.586363] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:04 2020][8592368.592040] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:04 2020][8592368.598755] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.603914] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:04 2020][8592368.609675] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:04 2020][8592368.616394] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.621463] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.627749] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.634375] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.640660] watchdog/19 S ffff9c7ae9849040 0 106 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.647949] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.650578] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:04 2020][8592368.656088] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.661243] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.667259] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.673279] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.678355] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.684657] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.691295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.697581] migration/19 S ffff9c7ae98b0000 0 107 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.704872] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.707501] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.712679] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.718688] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.724712] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.729787] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.736059] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.742692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.748962] ksoftirqd/19 S ffff9c7ae98b1040 0 108 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.756248] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.758890] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.764048] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.770072] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.776096] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.781168] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.787511] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.794152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.800445] kworker/19:0H S ffff9c7ae98b30c0 0 110 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.807743] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.810392] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:04 2020][8592368.816581] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.821741] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:04 2020][8592368.827490] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:04 2020][8592368.834211] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.839285] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.845553] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.852187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.858491] watchdog/20 S ffff9c4b6957e180 0 111 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.865769] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.868400] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:04 2020][8592368.873927] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.879070] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.885083] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.891113] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.896181] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.902451] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.909100] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.915374] migration/20 S ffff9c4b6957d140 0 112 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.922666] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.925296] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.930444] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.936472] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.942507] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.947561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.953866] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592368.960481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.966765] ksoftirqd/20 S ffff9c4b695c8000 0 113 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592368.974033] Call Trace: [Tue Mar 17 17:58:04 2020][8592368.976679] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592368.981820] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592368.987848] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592368.993857] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592368.998932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.005216] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.011875] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.018156] kworker/20:0H S ffff9c4b695ca080 0 115 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.025450] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.028098] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:04 2020][8592369.034299] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.039458] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:04 2020][8592369.045224] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:04 2020][8592369.051939] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.056995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.063276] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.069907] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.076188] watchdog/21 S ffff9c5ae952e180 0 116 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.083477] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.086120] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:04 2020][8592369.091627] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.096765] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.102788] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.108812] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.113882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.120164] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.126799] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.133085] migration/21 S ffff9c5ae952d140 0 117 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.140376] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.143017] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.148159] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.154166] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.160189] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.165257] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.171543] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.178170] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.184454] ksoftirqd/21 S ffff9c5ae952c100 0 118 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.191744] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.194380] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.199533] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.205541] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.211563] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.216633] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.222920] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.229554] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.235843] kworker/21:0H S ffff9c5ae952a080 0 120 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.243139] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.245765] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:04 2020][8592369.251943] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.257085] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:04 2020][8592369.262837] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:04 2020][8592369.269551] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.274606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.280889] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.287521] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.293801] watchdog/22 S ffff9c6ae99d6180 0 121 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.301090] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.303731] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:04 2020][8592369.309240] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.314379] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.320401] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.326422] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.331478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.337758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.344392] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.350676] migration/22 S ffff9c6ae99d5140 0 122 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.357961] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.360609] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:04 2020][8592369.366901] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.372047] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.378072] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.384100] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:04 2020][8592369.389160] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.395447] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:04 2020][8592369.402083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:04 2020][8592369.408370] ksoftirqd/22 S ffff9c6ae99d4100 0 123 2 0x00000000 [Tue Mar 17 17:58:04 2020][8592369.415657] Call Trace: [Tue Mar 17 17:58:04 2020][8592369.418306] [] schedule+0x29/0x70 [Tue Mar 17 17:58:04 2020][8592369.423470] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:04 2020][8592369.429490] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.435530] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.440589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.446925] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.453557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.459842] kworker/22:0H S ffff9c6ae99d2080 0 125 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.467132] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.471507] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:05 2020][8592369.477686] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.482858] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:05 2020][8592369.488608] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:05 2020][8592369.495361] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.500420] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.506725] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.513341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.519625] watchdog/23 S ffff9c7ae98b6180 0 126 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.526910] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.529540] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:05 2020][8592369.535051] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.540205] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.546216] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.552246] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.557318] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.563609] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.570251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.576537] migration/23 S ffff9c7ae98b5140 0 127 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.583809] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.586439] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:05 2020][8592369.592748] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.597899] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.603928] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.609954] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.615029] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.621318] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.627950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.634233] ksoftirqd/23 S ffff9c7ae98b4100 0 128 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.641520] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.644163] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.649324] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.655354] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.661386] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.666473] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.672759] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.679396] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.685692] kworker/23:0H S ffff9c7ae98e1040 0 130 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.692997] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.695649] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:05 2020][8592369.701836] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.707014] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:05 2020][8592369.712763] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:05 2020][8592369.719488] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.724542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.730850] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.737465] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.743773] watchdog/24 S ffff9c4b695ce180 0 131 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.751051] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.753698] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:05 2020][8592369.759194] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.764354] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.770365] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.776424] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.781485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.787793] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.794414] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.800706] migration/24 S ffff9c4b695cd140 0 132 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.808007] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.810657] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:05 2020][8592369.816931] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.822093] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.828102] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.834145] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.839206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.845509] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.852126] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.858416] ksoftirqd/24 S ffff9c4b695cc100 0 133 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.865686] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.868314] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.873463] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.879510] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.885518] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.890626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.896898] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.903533] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.909814] kworker/24:0H S ffff9c4b69618000 0 135 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.917110] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.919758] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:05 2020][8592369.925959] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.931118] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:05 2020][8592369.936887] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:05 2020][8592369.943613] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592369.948693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.954994] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592369.961629] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592369.967928] watchdog/25 S ffff9c5ae9570000 0 136 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592369.975233] Call Trace: [Tue Mar 17 17:58:05 2020][8592369.977878] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:05 2020][8592369.983390] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592369.988555] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592369.994597] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.000624] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.005699] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.011989] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.018662] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.024947] migration/25 S ffff9c5ae9571040 0 137 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.032253] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.034904] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.040065] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592370.046091] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.052119] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.057192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.063477] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.070112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.076423] ksoftirqd/25 S ffff9c5ae9572080 0 138 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.083713] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.086360] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.091512] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592370.097564] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.103573] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.108664] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.114936] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.121571] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.127854] kworker/25:0H S ffff9c5ae9574100 0 140 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.135147] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.137792] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:05 2020][8592370.143995] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.149154] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:05 2020][8592370.154919] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:05 2020][8592370.161634] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.166708] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.172990] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.179627] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.185912] watchdog/26 S ffff9c6ae9a18000 0 141 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.193226] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.195870] [] ? watchdog_cleanup+0x10/0x10 [Tue Mar 17 17:58:05 2020][8592370.201896] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:05 2020][8592370.207403] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.212564] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592370.218583] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.224597] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.229669] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.235957] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.242595] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.248886] migration/26 S ffff9c6ae9a19040 0 142 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.256174] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.258819] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:05 2020][8592370.265093] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.270245] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592370.276257] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.282290] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.287362] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.293650] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.300276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.306562] ksoftirqd/26 S ffff9c6ae9a1a080 0 143 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.313849] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.316495] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.321635] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:05 2020][8592370.327672] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.333682] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.338753] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.345039] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:05 2020][8592370.351674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.357954] kworker/26:0H S ffff9c6ae9a1c100 0 145 2 0x00000000 [Tue Mar 17 17:58:05 2020][8592370.365250] Call Trace: [Tue Mar 17 17:58:05 2020][8592370.367896] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:05 2020][8592370.373577] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:05 2020][8592370.380297] [] schedule+0x29/0x70 [Tue Mar 17 17:58:05 2020][8592370.385458] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:05 2020][8592370.391225] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:05 2020][8592370.397946] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:05 2020][8592370.403024] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:05 2020][8592370.409302] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.415939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.422231] watchdog/27 S ffff9c7ae98e6180 0 146 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.429525] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.432173] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:06 2020][8592370.437667] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.442846] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.448856] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.454901] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.459958] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.466250] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.472899] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.479190] migration/27 S ffff9c7ae98e5140 0 147 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.486502] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.489151] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.494299] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.500324] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.506333] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.511428] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.517706] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.524357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.530626] ksoftirqd/27 S ffff9c7ae98e4100 0 148 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.537936] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.540568] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.545731] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.551760] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.557768] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.562836] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.569204] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.575821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.582117] kworker/27:0H S ffff9c7ae98e2080 0 150 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.589412] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.592046] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:06 2020][8592370.598226] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.603405] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:06 2020][8592370.609153] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:06 2020][8592370.615868] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.620937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.627205] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.633835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.640116] watchdog/28 S ffff9c4b6961e180 0 151 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.647413] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.650059] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:06 2020][8592370.655549] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.660711] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.666717] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.672752] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.677815] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.684089] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.690724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.696992] migration/28 S ffff9c4b6961d140 0 152 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.704294] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.706924] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.712070] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.718093] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.724118] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.729179] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.735477] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.742094] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.748397] ksoftirqd/28 S ffff9c4b6961c100 0 153 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.755689] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.758355] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.763499] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.769538] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.775545] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.780616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.786897] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.793531] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.799819] kworker/28:0H S ffff9c4b6961a080 0 155 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.807122] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.809756] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:06 2020][8592370.815991] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.821135] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:06 2020][8592370.826907] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:06 2020][8592370.833627] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.838700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.844981] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.851615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.857905] watchdog/29 S ffff9c5ae9576180 0 156 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.865194] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.867835] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:06 2020][8592370.873340] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.878480] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.884503] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.890524] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.895602] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.901886] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.908518] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.914799] migration/29 S ffff9c5ae9575140 0 157 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.922089] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.924734] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:06 2020][8592370.931017] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.936157] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.942181] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.948200] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592370.953274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.959555] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592370.966188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.972467] ksoftirqd/29 S ffff9c5ae95a0000 0 158 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592370.979756] Call Trace: [Tue Mar 17 17:58:06 2020][8592370.982386] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592370.987541] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592370.993550] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592370.999581] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.004647] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.010937] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.017570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.023857] kworker/29:0H S ffff9c5ae95a2080 0 160 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.031145] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.033789] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:06 2020][8592371.039977] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.045155] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:06 2020][8592371.050905] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:06 2020][8592371.057634] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.062692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.068987] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.075607] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.081895] watchdog/30 S ffff9c6ae9a1e180 0 161 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.089183] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.091815] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:06 2020][8592371.097306] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.102463] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592371.108468] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.114491] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.119563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.125847] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.132481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.138752] migration/30 S ffff9c6ae9a1d140 0 162 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.146068] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.148701] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.153839] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592371.159870] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.165879] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.170978] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.177264] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.183929] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.190226] ksoftirqd/30 S ffff9c6ae9a48000 0 163 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.197536] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.200179] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.205366] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592371.211382] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.217432] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.222505] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.228805] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.235435] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.241741] kworker/30:0H S ffff9c6ae9a4a080 0 165 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.249039] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.251672] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:06 2020][8592371.257342] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:06 2020][8592371.264068] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.269212] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:06 2020][8592371.275024] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:06 2020][8592371.281727] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.286836] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.293124] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.299829] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.306124] watchdog/31 S ffff9c7ae9928000 0 166 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.313400] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.316045] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:06 2020][8592371.321534] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.326711] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592371.332753] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.338780] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.344493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.350784] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.357460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.363745] migration/31 S ffff9c7ae9929040 0 167 2 0x00000000 [Tue Mar 17 17:58:06 2020][8592371.371033] Call Trace: [Tue Mar 17 17:58:06 2020][8592371.373680] [] schedule+0x29/0x70 [Tue Mar 17 17:58:06 2020][8592371.378867] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:06 2020][8592371.384894] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.390947] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:06 2020][8592371.396020] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.402323] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:06 2020][8592371.408952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:06 2020][8592371.415240] ksoftirqd/31 S ffff9c7ae992a080 0 168 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.422530] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.425163] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.430310] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.436336] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.442349] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.447477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.453744] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.460365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.466672] kworker/31:0H S ffff9c7ae992c100 0 170 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.473949] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.476583] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:07 2020][8592371.482823] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.487983] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:07 2020][8592371.494177] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:07 2020][8592371.500911] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.506022] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.512312] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.518970] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.525259] watchdog/32 S ffff9c4b69619040 0 171 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.532574] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.535220] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:07 2020][8592371.540745] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.545906] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.551960] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.557999] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.563089] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.569371] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.576009] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.582294] migration/32 S ffff9c4b69688000 0 172 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.589601] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.592244] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.598723] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.604750] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.610799] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.615868] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.622179] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.628808] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.635113] ksoftirqd/32 S ffff9c4b69689040 0 173 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.642406] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.645071] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.650218] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.656296] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.662311] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.669342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.675627] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.682283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.688567] kworker/32:0H S ffff9c4b6968b0c0 0 175 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.695876] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.698510] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:07 2020][8592371.706130] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.711275] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:07 2020][8592371.717796] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:07 2020][8592371.724516] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.729617] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.735916] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.742578] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.748861] watchdog/33 S ffff9c5ae95a6180 0 176 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.760981] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.763645] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:07 2020][8592371.769134] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.774318] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.780333] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.786367] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.791444] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.797738] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.804386] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.810672] migration/33 S ffff9c5ae95a5140 0 177 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.817959] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.820591] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.825752] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.831791] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.837800] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.842881] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.849153] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.855782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.862062] ksoftirqd/33 S ffff9c5ae95a4100 0 178 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.869354] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.871997] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.877155] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.883177] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.889211] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.894278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.900586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.907222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.913535] kworker/33:0H S ffff9c5ae95d8000 0 180 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.920830] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.923465] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:07 2020][8592371.929675] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.934831] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:07 2020][8592371.940618] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:07 2020][8592371.947336] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592371.952419] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.958708] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592371.965339] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592371.971633] watchdog/34 S ffff9c6ae9a4e180 0 181 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592371.978945] Call Trace: [Tue Mar 17 17:58:07 2020][8592371.981574] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:07 2020][8592371.987125] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592371.992267] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592371.998305] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.004315] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.009396] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.015669] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.022289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.028585] migration/34 S ffff9c6ae9a4d140 0 182 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.035857] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.038488] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.043662] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.049688] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.055701] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.060774] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.067069] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.073684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.084114] ksoftirqd/34 S ffff9c6ae9a4c100 0 183 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.091414] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.094081] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.099226] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.105296] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.111306] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.116401] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.122668] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.129326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.135617] watchdog/35 S ffff9c7ae992e180 0 186 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.142903] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.145532] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:07 2020][8592372.154386] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.159544] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.165610] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.171634] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.176720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.183003] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.189650] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.195940] migration/35 S ffff9c7ae992d140 0 187 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.203240] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.205887] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.211057] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.217093] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.223145] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.228217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.234506] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.241137] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.247427] ksoftirqd/35 S ffff9c7ae9958000 0 188 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.254718] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.257365] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.262507] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.268566] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.274579] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.279679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.285964] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.292619] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.298888] kworker/35:0H S ffff9c7ae995a080 0 190 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.306162] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.308810] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:07 2020][8592372.315007] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.320175] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:07 2020][8592372.325945] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:07 2020][8592372.332658] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.337731] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.344014] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.350631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.356920] watchdog/36 S ffff9c4b6968e180 0 191 2 0x00000000 [Tue Mar 17 17:58:07 2020][8592372.364240] Call Trace: [Tue Mar 17 17:58:07 2020][8592372.366888] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:07 2020][8592372.372405] [] schedule+0x29/0x70 [Tue Mar 17 17:58:07 2020][8592372.377587] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:07 2020][8592372.383622] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.389651] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:07 2020][8592372.394725] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.401001] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:07 2020][8592372.407653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:07 2020][8592372.413933] migration/36 S ffff9c4b6968d140 0 192 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.421217] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.423862] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.429032] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.435094] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.441111] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.446187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.452496] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.459112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.465417] ksoftirqd/36 S ffff9c4b6968c100 0 193 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.472687] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.475332] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.480473] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.486538] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.492567] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.497665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.503948] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.510582] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.516871] kworker/36:0H S ffff9c4b69701040 0 195 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.524164] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.526811] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:08 2020][8592372.533015] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.538160] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:08 2020][8592372.543980] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:08 2020][8592372.550686] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.555782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.562082] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.568739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.575026] watchdog/37 S ffff9c5ae95de180 0 196 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.582322] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.584980] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:08 2020][8592372.590476] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.595636] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.601644] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.607660] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.612721] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.619014] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.625633] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.631921] migration/37 S ffff9c5ae95dd140 0 197 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.639193] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.641861] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.647004] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.653024] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.659032] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.664102] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.670371] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.676992] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.683290] ksoftirqd/37 S ffff9c5ae95dc100 0 198 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.690560] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.693190] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.698342] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.704381] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.710395] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.715490] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.721780] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.728413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.734699] kworker/37:0H S ffff9c5ae95da080 0 200 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.741994] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.744642] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:08 2020][8592372.750861] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.756020] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:08 2020][8592372.761797] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:08 2020][8592372.768517] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.775953] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.782237] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.788882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.795167] watchdog/38 S ffff9c6ae9a86180 0 201 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.802456] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.805100] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:08 2020][8592372.810609] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.815767] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.821795] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.827817] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.832888] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.839168] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.845817] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.852098] migration/38 S ffff9c6ae9a85140 0 202 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.859389] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.862032] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.867191] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.873213] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.879238] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.884303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.890591] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.897218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.903502] ksoftirqd/38 S ffff9c6ae9a84100 0 203 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.910790] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.913418] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.918576] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592372.924603] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.930629] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.935696] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.941984] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592372.948614] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592372.954897] kworker/38:0H S ffff9c6ae9a82080 0 205 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592372.962188] Call Trace: [Tue Mar 17 17:58:08 2020][8592372.964838] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:08 2020][8592372.971017] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592372.976173] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:08 2020][8592372.981923] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:08 2020][8592372.990705] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592372.995777] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.002074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.008694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.014981] watchdog/39 S ffff9c7ae995e180 0 206 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.022295] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.024941] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:08 2020][8592373.030427] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.035616] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.041626] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.047650] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.052723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.059013] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.065646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.071931] migration/39 S ffff9c7ae995d140 0 207 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.079219] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.081866] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.087017] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.093062] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.099071] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.104148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.110427] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.117083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.123367] ksoftirqd/39 S ffff9c7ae995c100 0 208 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.130656] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.133288] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.138438] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.144465] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.150479] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.155555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.161848] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.168477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.174749] kworker/39:0H S ffff9c7ae99b0000 0 210 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.182064] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.184698] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:08 2020][8592373.190885] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.196064] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:08 2020][8592373.201813] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:08 2020][8592373.208528] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.213600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.219893] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.226528] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.232834] watchdog/40 S ffff9c4b69706180 0 211 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.240117] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.242775] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:08 2020][8592373.248267] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.253412] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.259423] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.265435] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.270502] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.276793] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.283415] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.289739] migration/40 S ffff9c4b69705140 0 212 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.297025] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.299686] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.304825] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.310845] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.316854] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.321924] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.328207] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.334840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.341126] ksoftirqd/40 S ffff9c4b69704100 0 213 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.348417] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.351060] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.356202] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:08 2020][8592373.362217] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.368260] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:08 2020][8592373.373334] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.379645] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:08 2020][8592373.386273] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:08 2020][8592373.392581] kworker/40:0H S ffff9c4b69702080 0 215 2 0x00000000 [Tue Mar 17 17:58:08 2020][8592373.399889] Call Trace: [Tue Mar 17 17:58:08 2020][8592373.402518] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:08 2020][8592373.408730] [] schedule+0x29/0x70 [Tue Mar 17 17:58:08 2020][8592373.413872] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:08 2020][8592373.419643] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:08 2020][8592373.426356] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.431428] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.437709] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.444345] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.450636] watchdog/41 S ffff9c5ae95d9040 0 216 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.457931] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.460562] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:09 2020][8592373.466085] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.471242] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.477267] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.483288] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.488360] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.494639] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.501274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.507556] migration/41 S ffff9c5ae9638000 0 217 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.514870] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.517499] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.522652] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.528660] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.534705] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.539779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.546073] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.552707] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.558999] ksoftirqd/41 S ffff9c5ae9639040 0 218 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.566288] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.568945] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.574093] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.580149] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.586171] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.591258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.597541] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.604200] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.610482] kworker/41:0H S ffff9c5ae963b0c0 0 220 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.617784] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.620417] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:09 2020][8592373.626646] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.631787] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:09 2020][8592373.637582] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:09 2020][8592373.644303] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.649390] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.655683] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.662321] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.668615] watchdog/42 S ffff9c6ae9a81040 0 221 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.675914] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.678546] [] ? touch_nmi_watchdog+0x30/0x30 [Tue Mar 17 17:58:09 2020][8592373.684792] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:09 2020][8592373.690280] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.695454] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.701481] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.707510] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.712586] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.718911] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.725543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.731857] migration/42 S ffff9c6ae9ae8000 0 222 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.739149] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.741798] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.746942] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.752991] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.759010] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.764084] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.770364] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.776998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.783278] ksoftirqd/42 S ffff9c6ae9ae9040 0 223 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.790548] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.793194] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.798335] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.804355] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.810390] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.815444] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.822356] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.828973] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.835292] watchdog/43 S ffff9c7ae99b6180 0 226 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.842583] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.848669] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:09 2020][8592373.854158] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.859344] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.865355] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.871389] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.876475] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.882760] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.889393] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.895679] migration/43 S ffff9c7ae99b5140 0 227 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.902967] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.905600] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:09 2020][8592373.911896] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.917080] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.923092] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.929134] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.934212] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.940481] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.947116] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.953427] ksoftirqd/43 S ffff9c7ae99b4100 0 228 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592373.960720] Call Trace: [Tue Mar 17 17:58:09 2020][8592373.963381] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592373.968522] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592373.974561] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.980571] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592373.985648] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592373.991933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592373.998574] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.004863] kworker/43:0H S ffff9c7ae99b2080 0 230 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.012138] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.014789] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:09 2020][8592374.021022] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.026181] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:09 2020][8592374.031963] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:09 2020][8592374.038680] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.043753] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.050053] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.056712] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.062997] watchdog/44 S ffff9c4b69768000 0 231 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.070285] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.072929] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:09 2020][8592374.078437] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.083606] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592374.089652] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.095680] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.100776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.107074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.113735] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.120024] migration/44 S ffff9c4b69769040 0 232 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.127348] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.129993] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.137303] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592374.143332] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.149396] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.154468] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.160781] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.167429] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.173744] ksoftirqd/44 S ffff9c4b6976a080 0 233 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.181038] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.183697] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.188842] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592374.194908] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.200915] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.205974] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.212254] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.218870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.225136] kworker/44:0H S ffff9c4b6976c100 0 235 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.232446] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.235111] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:09 2020][8592374.241292] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.246469] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:09 2020][8592374.252222] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:09 2020][8592374.258949] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.264022] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.270293] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.276927] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.283209] watchdog/45 S ffff9c5ae963e180 0 236 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.290481] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.293125] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:09 2020][8592374.298653] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.303809] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592374.309855] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.315883] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.320960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.327231] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.333888] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.340176] migration/45 S ffff9c5ae963d140 0 237 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.347474] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.350118] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:09 2020][8592374.360450] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.365590] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:09 2020][8592374.371641] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.377667] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:09 2020][8592374.382742] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.389022] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:09 2020][8592374.395674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:09 2020][8592374.401977] ksoftirqd/45 S ffff9c5ae963c100 0 238 2 0x00000000 [Tue Mar 17 17:58:09 2020][8592374.409286] Call Trace: [Tue Mar 17 17:58:09 2020][8592374.411929] [] schedule+0x29/0x70 [Tue Mar 17 17:58:09 2020][8592374.417113] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.423135] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.429195] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.434263] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.440566] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.447202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.453517] kworker/45:0H S ffff9c5ae9671040 0 240 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.460825] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.463456] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:10 2020][8592374.469643] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.474814] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:10 2020][8592374.480564] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:10 2020][8592374.487286] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.492354] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.498640] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.505269] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.511565] watchdog/46 S ffff9c6ae9aee180 0 241 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.518858] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.521489] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:10 2020][8592374.527032] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.532216] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.538270] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.544297] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.549374] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.555662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.562299] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.568589] migration/46 S ffff9c6ae9aed140 0 242 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.575878] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.578509] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:10 2020][8592374.584827] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.589972] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.595993] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.602017] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.607086] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.613369] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.620020] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.626307] ksoftirqd/46 S ffff9c6ae9aec100 0 243 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.633597] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.636257] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.641402] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.647416] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.653480] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.658535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.664825] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.671451] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.677722] kworker/46:0H S ffff9c6ae9b21040 0 245 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.685045] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.687675] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:10 2020][8592374.693342] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:10 2020][8592374.700098] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.705242] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:10 2020][8592374.711037] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:10 2020][8592374.717748] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.722825] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.729102] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.735739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.742027] watchdog/47 S ffff9c7ae99b1040 0 246 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.749333] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.751981] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:10 2020][8592374.757494] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.762654] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.768678] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.774701] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.779762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.786059] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.792702] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.798987] migration/47 S ffff9c7ae99f0000 0 247 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.806274] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.808934] [] ? cpu_stop_queue_work+0xe0/0xe0 [Tue Mar 17 17:58:10 2020][8592374.815218] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.820376] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.826400] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.832430] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.837486] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.843776] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.850412] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.856696] ksoftirqd/47 S ffff9c7ae99f1040 0 248 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.863969] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.866601] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.871799] [] smpboot_thread_fn+0xe2/0x1a0 [Tue Mar 17 17:58:10 2020][8592374.877827] [] ? lg_double_unlock+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.883855] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.888929] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.895217] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.901851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.908139] kworker/47:0H S ffff9c7ae99f30c0 0 250 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.915429] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.918091] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:10 2020][8592374.924278] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.929474] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:10 2020][8592374.935231] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:10 2020][8592374.941972] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.947030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.953314] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592374.959945] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592374.966243] kdevtmpfs S ffff9c4b6905a080 0 255 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592374.973532] Call Trace: [Tue Mar 17 17:58:10 2020][8592374.976180] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592374.981327] [] devtmpfsd+0x16d/0x180 [Tue Mar 17 17:58:10 2020][8592374.986749] [] ? handle_create.isra.2+0x220/0x220 [Tue Mar 17 17:58:10 2020][8592374.993277] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592374.998348] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.004627] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.011245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.017530] netns S ffff9c4b6905b0c0 0 256 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.024827] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.027459] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.032623] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.038470] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.044654] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.049706] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.056019] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.062653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.068937] khungtaskd S ffff9c4b6905c100 0 257 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.076207] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.078850] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.083993] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:10 2020][8592375.090038] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:58:10 2020][8592375.095706] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:10 2020][8592375.102257] [] schedule_timeout_interruptible+0x1e/0x20 [Tue Mar 17 17:58:10 2020][8592375.109325] [] watchdog+0x4c/0x2c0 [Tue Mar 17 17:58:10 2020][8592375.114579] [] ? reset_hung_task_detector+0x20/0x20 [Tue Mar 17 17:58:10 2020][8592375.121312] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.126373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.132662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.139295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.145560] writeback S ffff9c4b6905d140 0 258 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.152877] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.155510] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.160678] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.166534] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.172739] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.177807] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.187403] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.194052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.200326] kintegrityd S ffff9c4b6905e180 0 259 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.207651] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.210285] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.215457] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.221319] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.227524] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.232597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.238884] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.245520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.251808] bioset S ffff9c4b69058000 0 260 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.259105] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.261754] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.266896] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.272767] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.278955] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.284023] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.290360] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.296979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.303285] bioset S ffff9c4b69059040 0 261 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.310580] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.313245] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.318399] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.324283] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.330467] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.335560] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.341830] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.348483] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.354773] bioset S ffff9c5a3eff0000 0 262 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.362047] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.364698] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.369872] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:10 2020][8592375.375723] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:10 2020][8592375.381931] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:10 2020][8592375.387019] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.393318] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:10 2020][8592375.399952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:10 2020][8592375.406249] kblockd S ffff9c5a3eff1040 0 263 2 0x00000000 [Tue Mar 17 17:58:10 2020][8592375.413547] Call Trace: [Tue Mar 17 17:58:10 2020][8592375.416198] [] schedule+0x29/0x70 [Tue Mar 17 17:58:10 2020][8592375.421345] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:11 2020][8592375.427207] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:11 2020][8592375.433390] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.438463] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.444753] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.451402] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.457689] md S ffff9c5a3eff2080 0 264 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.464981] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.467622] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.472784] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:11 2020][8592375.478636] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:11 2020][8592375.484840] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.489911] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.496216] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.502842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.509136] edac-poller S ffff9c5a3eff30c0 0 265 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.516429] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.519080] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.524219] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:11 2020][8592375.530094] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:11 2020][8592375.536273] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.541345] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.547631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.554265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.560532] watchdogd S ffff9c5a3eff4100 0 266 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.567822] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.570458] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.575647] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:11 2020][8592375.581505] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:11 2020][8592375.587722] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.592795] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.599090] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.605716] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.612007] irq/26-AMD-Vi S ffff9c5a3eff6180 0 268 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.619308] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.621957] [] ? irq_finalize_oneshot.part.31+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.629007] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.634161] [] irq_thread+0xd1/0x180 [Tue Mar 17 17:58:11 2020][8592375.639567] [] ? irq_forced_thread_fn+0x50/0x50 [Tue Mar 17 17:58:11 2020][8592375.645927] [] ? irq_thread_check_affinity+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.652747] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.657809] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.666085] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.672734] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.679053] irq/27-AMD-Vi S ffff9c5a3a118000 0 269 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.686327] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.688973] [] ? irq_finalize_oneshot.part.31+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.696019] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.701192] [] irq_thread+0xd1/0x180 [Tue Mar 17 17:58:11 2020][8592375.706593] [] ? irq_forced_thread_fn+0x50/0x50 [Tue Mar 17 17:58:11 2020][8592375.712996] [] ? irq_thread_check_affinity+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.719784] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.729103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.735448] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.742075] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.748391] irq/28-AMD-Vi S ffff9c5a3a119040 0 270 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.755672] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.758314] [] ? irq_finalize_oneshot.part.31+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.765393] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.770547] [] irq_thread+0xd1/0x180 [Tue Mar 17 17:58:11 2020][8592375.775959] [] ? irq_forced_thread_fn+0x50/0x50 [Tue Mar 17 17:58:11 2020][8592375.782410] [] ? irq_thread_check_affinity+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.789210] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.794284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.800573] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.807206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.813492] irq/29-AMD-Vi S ffff9c5a3a11a080 0 271 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.820784] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.823414] [] ? irq_finalize_oneshot.part.31+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.830521] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.835665] [] irq_thread+0xd1/0x180 [Tue Mar 17 17:58:11 2020][8592375.841111] [] ? irq_forced_thread_fn+0x50/0x50 [Tue Mar 17 17:58:11 2020][8592375.847485] [] ? irq_thread_check_affinity+0xf0/0xf0 [Tue Mar 17 17:58:11 2020][8592375.861239] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592375.866300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.872621] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592375.879259] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592375.885545] kswapd0 D ffff9c5a3a11c100 0 276 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592375.892830] Call Trace: [Tue Mar 17 17:58:11 2020][8592375.895482] [] ? start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:58:11 2020][8592375.902376] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592375.909010] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:58:11 2020][8592375.915387] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:58:11 2020][8592375.920738] [] dquot_release+0x72/0x110 [Tue Mar 17 17:58:11 2020][8592375.926431] [] ldiskfs_release_dquot+0x6f/0xa0 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592375.933580] [] dqput+0xbd/0x1f0 [Tue Mar 17 17:58:11 2020][8592375.938566] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 17:58:11 2020][8592375.944066] [] dquot_drop+0x35/0x40 [Tue Mar 17 17:58:11 2020][8592375.949408] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592375.956389] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592375.963455] [] evict+0xb4/0x180 [Tue Mar 17 17:58:11 2020][8592375.968436] [] dispose_list+0x3e/0x50 [Tue Mar 17 17:58:11 2020][8592375.973946] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 17:58:11 2020][8592375.979886] [] prune_super+0x173/0x190 [Tue Mar 17 17:58:11 2020][8592375.985480] [] shrink_slab+0x175/0x340 [Tue Mar 17 17:58:11 2020][8592375.991077] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 17:58:11 2020][8592375.997362] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.003129] [] kswapd+0x173/0x440 [Tue Mar 17 17:58:11 2020][8592376.008290] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:11 2020][8592376.014308] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.020248] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592376.025315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.031605] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592376.038252] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.044543] kswapd1 D ffff9c5a3a11b0c0 0 277 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592376.051810] Call Trace: [Tue Mar 17 17:58:11 2020][8592376.054459] [] ? start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:58:11 2020][8592376.061372] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592376.067988] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:58:11 2020][8592376.074367] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:58:11 2020][8592376.079700] [] dquot_release+0x72/0x110 [Tue Mar 17 17:58:11 2020][8592376.085388] [] ldiskfs_release_dquot+0x6f/0xa0 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.092537] [] dqput+0xbd/0x1f0 [Tue Mar 17 17:58:11 2020][8592376.097520] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 17:58:11 2020][8592376.103040] [] dquot_drop+0x35/0x40 [Tue Mar 17 17:58:11 2020][8592376.108385] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.115375] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.122424] [] evict+0xb4/0x180 [Tue Mar 17 17:58:11 2020][8592376.127412] [] dispose_list+0x3e/0x50 [Tue Mar 17 17:58:11 2020][8592376.132920] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 17:58:11 2020][8592376.138860] [] prune_super+0x173/0x190 [Tue Mar 17 17:58:11 2020][8592376.144456] [] shrink_slab+0x175/0x340 [Tue Mar 17 17:58:11 2020][8592376.150051] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 17:58:11 2020][8592376.156333] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.162096] [] kswapd+0x173/0x440 [Tue Mar 17 17:58:11 2020][8592376.167255] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:11 2020][8592376.173275] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.179215] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592376.184280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.190552] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592376.197188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.203492] kswapd2 D ffff9c5a3a250000 0 278 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592376.210768] Call Trace: [Tue Mar 17 17:58:11 2020][8592376.213400] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 17:58:11 2020][8592376.219462] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592376.224637] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:58:11 2020][8592376.231715] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:11 2020][8592376.237741] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:58:11 2020][8592376.244982] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:58:11 2020][8592376.251702] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:58:11 2020][8592376.257904] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:58:11 2020][8592376.264717] [] ? ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.272050] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.279638] [] ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.286797] [] dqput+0xbd/0x1f0 [Tue Mar 17 17:58:11 2020][8592376.291783] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 17:58:11 2020][8592376.297291] [] dquot_drop+0x35/0x40 [Tue Mar 17 17:58:11 2020][8592376.302623] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.309623] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 17:58:11 2020][8592376.316695] [] evict+0xb4/0x180 [Tue Mar 17 17:58:11 2020][8592376.324431] [] dispose_list+0x3e/0x50 [Tue Mar 17 17:58:11 2020][8592376.329935] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 17:58:11 2020][8592376.335889] [] prune_super+0x173/0x190 [Tue Mar 17 17:58:11 2020][8592376.341492] [] shrink_slab+0x175/0x340 [Tue Mar 17 17:58:11 2020][8592376.347096] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 17:58:11 2020][8592376.353378] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.359145] [] kswapd+0x173/0x440 [Tue Mar 17 17:58:11 2020][8592376.364309] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:11 2020][8592376.370315] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 17:58:11 2020][8592376.376250] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:11 2020][8592376.381325] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.387599] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:11 2020][8592376.394233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:11 2020][8592376.400503] kswapd3 D ffff9c5a3a251040 0 279 2 0x00000000 [Tue Mar 17 17:58:11 2020][8592376.407806] Call Trace: [Tue Mar 17 17:58:11 2020][8592376.410438] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 17:58:11 2020][8592376.416487] [] schedule+0x29/0x70 [Tue Mar 17 17:58:11 2020][8592376.421639] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:58:12 2020][8592376.428708] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:12 2020][8592376.434738] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:58:12 2020][8592376.441987] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:58:12 2020][8592376.448706] [] ? unfreeze_partials.isra.42+0xd2/0x130 [Tue Mar 17 17:58:12 2020][8592376.455601] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:58:12 2020][8592376.461818] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:58:12 2020][8592376.468640] [] ? ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 17:58:12 2020][8592376.475965] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:58:12 2020][8592376.483565] [] ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 17:58:12 2020][8592376.490716] [] dqput+0xbd/0x1f0 [Tue Mar 17 17:58:12 2020][8592376.495710] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 17:58:12 2020][8592376.501217] [] dquot_drop+0x35/0x40 [Tue Mar 17 17:58:12 2020][8592376.506543] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 17:58:12 2020][8592376.513530] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 17:58:12 2020][8592376.520583] [] evict+0xb4/0x180 [Tue Mar 17 17:58:12 2020][8592376.525582] [] dispose_list+0x3e/0x50 [Tue Mar 17 17:58:12 2020][8592376.531117] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 17:58:12 2020][8592376.537054] [] prune_super+0x173/0x190 [Tue Mar 17 17:58:12 2020][8592376.542657] [] shrink_slab+0x175/0x340 [Tue Mar 17 17:58:12 2020][8592376.548250] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 17:58:12 2020][8592376.554538] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 17:58:12 2020][8592376.560305] [] kswapd+0x173/0x440 [Tue Mar 17 17:58:12 2020][8592376.565480] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:12 2020][8592376.571503] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 17:58:12 2020][8592376.577443] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.582530] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.588812] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.595438] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.601752] ksmd S ffff9c5a3a252080 0 280 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.609039] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.611712] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.616866] [] ksm_scan_thread+0x16a/0x240 [Tue Mar 17 17:58:12 2020][8592376.622839] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:12 2020][8592376.628857] [] ? ksm_do_scan+0x11b0/0x11b0 [Tue Mar 17 17:58:12 2020][8592376.634828] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.639902] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.646199] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.652837] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.659146] khugepaged S ffff9c5a3a2530c0 0 281 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.666448] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.669103] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.674252] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:12 2020][8592376.680313] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:12 2020][8592376.686845] [] khugepaged+0x226/0x480 [Tue Mar 17 17:58:12 2020][8592376.692414] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:12 2020][8592376.698426] [] ? khugepaged_scan_mm_slot+0xf00/0xf00 [Tue Mar 17 17:58:12 2020][8592376.705257] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.710328] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.716644] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.723278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.729622] crypto S ffff9c5a3a254100 0 282 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.736921] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.739559] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.744748] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592376.750602] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592376.756787] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.761876] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.768146] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.774786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.781060] kthrotld S ffff9c5a3a2b5140 0 290 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.788403] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.791038] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.796224] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592376.802085] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592376.808311] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.813369] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.819668] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.826297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.832587] kmpath_rdacd S ffff9c5a3a2b4100 0 295 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.839881] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.842546] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.847689] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592376.853544] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592376.859726] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.864793] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.871075] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.877693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.883981] kaluad S ffff9c5a3a2b30c0 0 296 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.891301] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.893947] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.899090] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592376.904939] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592376.911166] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.916223] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.922505] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.929133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.935460] kpsmoused S ffff9c5a3a2b1040 0 299 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.942753] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.945382] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592376.950528] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592376.956416] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592376.962600] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592376.967677] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.973961] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592376.980576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592376.986878] ipv6_addrconf S ffff9c7ae99f4100 0 301 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592376.994210] Call Trace: [Tue Mar 17 17:58:12 2020][8592376.996842] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.002020] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.007874] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592377.014055] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.019123] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.025463] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.032085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.038416] deferwq S ffff9c5a39d50000 0 315 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.045708] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.048377] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.053524] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.059414] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592377.065599] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.070700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.076997] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.083675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.089977] kauditd S ffff9c5a39ea2080 0 442 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.097280] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.099921] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.105119] [] kauditd_thread+0x184/0x1e0 [Tue Mar 17 17:58:12 2020][8592377.110975] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:12 2020][8592377.116787] [] ? audit_printk_skb+0x70/0x70 [Tue Mar 17 17:58:12 2020][8592377.122811] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.127919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.134210] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.140895] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.147183] scsi_eh_0 S ffff9c7a37a29040 0 579 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.154516] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.157160] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.162348] [] scsi_error_handler+0xa3/0x8b0 [Tue Mar 17 17:58:12 2020][8592377.168459] [] ? scsi_eh_get_sense+0x250/0x250 [Tue Mar 17 17:58:12 2020][8592377.174756] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.179829] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.186116] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.192751] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.199022] scsi_tmf_0 S ffff9c5a39ea6180 0 580 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.206321] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.208952] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.214093] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.220018] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592377.226201] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.231292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.237576] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.244211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.250497] ata_sff S ffff9c7a39829900 0 1557 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.261080] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.263708] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.268882] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.274745] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592377.280966] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.286038] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.292360] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.298987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.307667] scsi_eh_1 S ffff9c7a36a28000 0 1958 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.314960] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.317637] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.322784] [] scsi_error_handler+0xa3/0x8b0 [Tue Mar 17 17:58:12 2020][8592377.328900] [] ? scsi_eh_get_sense+0x250/0x250 [Tue Mar 17 17:58:12 2020][8592377.335171] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.340257] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.346531] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.353214] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.359499] scsi_tmf_1 S ffff9c6a37fe0000 0 1961 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.366821] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.369457] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.376380] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.382219] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:12 2020][8592377.388434] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:12 2020][8592377.393508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.399814] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:12 2020][8592377.406450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:12 2020][8592377.412727] mlx5_health0000 S ffff9c6a37fe2080 0 2111 2 0x00000000 [Tue Mar 17 17:58:12 2020][8592377.420034] Call Trace: [Tue Mar 17 17:58:12 2020][8592377.422667] [] schedule+0x29/0x70 [Tue Mar 17 17:58:12 2020][8592377.427818] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:12 2020][8592377.433671] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.439852] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.444929] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.451231] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.457843] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.464140] mlx5_page_alloc S ffff9c6a37fe30c0 0 2113 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.471417] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.474071] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.479217] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.485090] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.491275] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.496368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.502643] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.509323] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.515593] mlx5_cmd_0000:0 S ffff9c7a32a8c100 0 2657 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.522915] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.525549] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.530697] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.536553] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.542733] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.547788] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.554110] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.560726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.567055] scsi_eh_2 S ffff9c6a341fc100 0 2671 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.574343] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.577024] [] ? __scsi_iterate_devices+0x68/0x90 [Tue Mar 17 17:58:13 2020][8592377.583560] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.588885] [] scsi_error_handler+0xa3/0x8b0 [Tue Mar 17 17:58:13 2020][8592377.594987] [] ? scsi_eh_get_sense+0x250/0x250 [Tue Mar 17 17:58:13 2020][8592377.601275] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.606332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.614037] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.620656] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.626963] scsi_tmf_2 S ffff9c6a341f9040 0 2676 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.634257] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.636916] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.642063] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.647954] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.654141] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.659246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.665536] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.672177] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.678462] mlx5_events S ffff9c6a2e618000 0 4715 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.685793] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.688435] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.693650] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.699505] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.705725] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.710795] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.717085] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.723715] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.730005] mlx5_fw_tracer S ffff9c5a2ab89040 0 4717 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.737302] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.739955] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.745103] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.751028] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.757212] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.762315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.768589] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.775238] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.781524] mlx5_fc S ffff9c5a2ab8b0c0 0 4719 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.788827] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.791460] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.797167] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.803004] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.809246] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.814336] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.820651] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.827300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.839870] poll_mpt3sas0_h S ffff9c5a2ab8c100 0 4721 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.847202] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.849838] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.854985] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.860885] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.867072] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.872180] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.878450] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.885107] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.891396] kworker/31:1 S ffff9c8a2d8ba080 0 4738 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.898690] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.901336] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:13 2020][8592377.907554] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.912715] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.918515] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:13 2020][8592377.925254] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.930340] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.936638] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.943330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.949629] ib-comp-wq S ffff9c8a2d8f5140 0 4743 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592377.956966] Call Trace: [Tue Mar 17 17:58:13 2020][8592377.959649] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592377.964859] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592377.970713] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592377.976929] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592377.982004] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592377.988325] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592377.994964] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.001239] kworker/u261:0 S ffff9c8a2d8f4100 0 4744 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.008590] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.011227] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.016376] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.022176] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:13 2020][8592378.028877] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.033979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.040257] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.046882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.053170] kworker/u263:0 S ffff9c6a2e61b0c0 0 4746 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.060514] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.063162] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.068302] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.074069] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:13 2020][8592378.080774] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.085855] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.092132] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.098767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.105054] kworker/u264:0 S ffff9c7a27d430c0 0 4747 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.112330] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.114979] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.120176] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.125944] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:13 2020][8592378.132696] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.137808] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.144123] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.150740] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.157073] kworker/u265:0 S ffff9c8a2d8f1040 0 4748 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.164378] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.167052] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.172202] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.177989] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:13 2020][8592378.184698] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.189774] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.196041] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.202690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.208977] ib-comp-unb-wq S ffff9c8a2d8f2080 0 4749 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.216268] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.218909] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.224051] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.229885] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592378.236087] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.241143] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.247418] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.254065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.260406] ib_mcast S ffff9c8a2d8f30c0 0 4750 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.267684] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.270346] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.275516] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.281359] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592378.287547] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.292650] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.298928] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.305595] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.311863] ib_nl_sa_wq S ffff9c6a2e61c100 0 4751 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.319190] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.321825] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.327016] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.332879] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592378.339108] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.344191] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.350465] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.357103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.363450] mlx5_ib_sigerr_ S ffff9c6a2fe75140 0 4763 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.370733] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.373368] [] schedule+0x29/0x70 [Tue Mar 17 17:58:13 2020][8592378.378603] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:13 2020][8592378.384440] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:13 2020][8592378.390669] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:13 2020][8592378.395747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.402057] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:13 2020][8592378.408697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:13 2020][8592378.414994] ib_mad1 S ffff9c6a2fe74100 0 4765 2 0x00000000 [Tue Mar 17 17:58:13 2020][8592378.422289] Call Trace: [Tue Mar 17 17:58:13 2020][8592378.424947] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.430098] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.435978] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:14 2020][8592378.442172] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.447320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.453594] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.460245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.466526] to_fifo S ffff9c6a2fe730c0 0 4766 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.473837] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.476472] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.481635] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.487468] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:14 2020][8592378.493702] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.498778] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.505119] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.511762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.518060] mkey_cache S ffff9c6a2fe72080 0 4767 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.525366] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.528013] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.533163] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.539037] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:14 2020][8592378.545222] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.550324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.556593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.563240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.569535] kworker/8:1H S ffff9c5a2976e180 0 4771 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.576861] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.579506] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592378.585759] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.590907] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.596722] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.603448] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.608545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.614830] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.621517] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.627808] kworker/2:1H S ffff9c7a35a82080 0 4773 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.635118] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.637764] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592378.644002] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.649162] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.654948] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.661666] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.666757] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.673049] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.679714] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.686003] poll_mpt3sas0_s S ffff9c5a2976c100 0 4777 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.693333] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.695982] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.701184] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.707050] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:14 2020][8592378.713287] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.718356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.724666] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.731280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.737586] kworker/43:1H S ffff9c8a31548000 0 5584 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.744881] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.747515] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592378.753748] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.758913] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.764691] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.771416] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.776496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.782772] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.789391] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.795682] kworker/47:1H S ffff9c8a31549040 0 5585 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.803020] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.805652] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592378.811888] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.817056] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.822811] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.829535] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.834597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.840888] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.847551] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.853821] kworker/42:1H S ffff9c7a30dcd140 0 5587 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.861148] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.863779] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:14 2020][8592378.870577] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.877298] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.882482] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.888250] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.894994] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.900045] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.906350] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.912982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.919285] kworker/35:1H S ffff9c8a3154e180 0 5588 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.926578] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.929213] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592378.935421] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592378.940575] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592378.946358] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.953073] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592378.958151] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.964437] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592378.971071] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592378.977349] kworker/34:1H S ffff9c7a30dca080 0 5589 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592378.984652] Call Trace: [Tue Mar 17 17:58:14 2020][8592378.987302] [] ? start_worker+0x27/0x30 [Tue Mar 17 17:58:14 2020][8592378.993019] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 17:58:14 2020][8592378.999735] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.004880] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.010649] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.017367] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.022420] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.028748] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.035369] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.041672] kworker/39:1H S ffff9c8a3154a080 0 5590 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.048971] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.051604] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.057830] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.062987] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.068762] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.075462] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.080539] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.086821] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.093453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.099735] kworker/31:1H S ffff9c8a3154d140 0 5592 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.107027] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.109676] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.115940] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.121097] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.126869] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.133586] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.138661] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.144947] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.151599] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.157878] kworker/23:1H S ffff9c8a3154b0c0 0 5594 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.165173] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.167819] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.174002] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.179164] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.184953] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.191654] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.196758] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.203029] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.213727] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.219996] kworker/22:1H S ffff9c7a32888000 0 5595 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.227303] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.229942] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.236164] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.241324] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.247108] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.253814] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.258913] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.265210] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.271871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.278155] kworker/27:1H S ffff9c8a3154c100 0 5596 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.285475] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.288123] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.294359] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.299520] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.305313] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.312022] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.317120] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.323407] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.334179] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.340495] kworker/19:1H S ffff9c8a31bd8000 0 5598 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.347775] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.350448] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.356644] [] schedule+0x29/0x70 [Tue Mar 17 17:58:14 2020][8592379.361824] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:14 2020][8592379.367573] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:14 2020][8592379.374335] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:14 2020][8592379.379394] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.385703] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:14 2020][8592379.392331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:14 2020][8592379.398668] kworker/15:1H S ffff9c8a31bde180 0 5600 2 0x00000000 [Tue Mar 17 17:58:14 2020][8592379.405959] Call Trace: [Tue Mar 17 17:58:14 2020][8592379.408597] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:14 2020][8592379.414826] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.419986] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.425770] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:15 2020][8592379.432472] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.437533] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.443809] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.450457] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.456741] kworker/11:1H S ffff9c8a31bd9040 0 5602 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.464043] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.466691] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:15 2020][8592379.472918] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.478081] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.483856] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:15 2020][8592379.490580] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.495676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.502004] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.508670] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.514959] kworker/7:1H S ffff9c8a31bda080 0 5603 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.522235] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.524868] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:15 2020][8592379.531110] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.536254] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.542056] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:15 2020][8592379.548761] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.553849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.560135] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.566773] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.573056] kworker/6:1H R running task 0 5604 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.580378] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.583023] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:15 2020][8592379.591225] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.596380] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.603931] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:15 2020][8592379.610645] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.615750] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.622034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.629314] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.635598] kworker/3:1H S ffff9c8a31bdd140 0 5605 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.642899] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.645532] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:15 2020][8592379.652363] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.657509] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.664777] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:15 2020][8592379.671498] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.676583] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.682867] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.692989] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.699257] jbd2/sda2-8 S ffff9c6a336e4100 0 5656 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.706570] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.709215] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.714402] [] kjournald2+0x203/0x260 [jbd2] [Tue Mar 17 17:58:15 2020][8592379.720513] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:15 2020][8592379.726561] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:58:15 2020][8592379.733021] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.738110] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.744390] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.751048] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.757330] ext4-rsv-conver S ffff9c6a336e30c0 0 5657 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.764652] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.767296] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.772469] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:15 2020][8592379.778322] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:15 2020][8592379.784566] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592379.789627] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.795926] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592379.802538] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592379.809847] systemd-journal S ffff9c6a30a24100 0 5729 1 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.817139] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.819806] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 17:58:15 2020][8592379.825646] [] ? schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.830996] [] ? schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:15 2020][8592379.838396] [] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0 [Tue Mar 17 17:58:15 2020][8592379.845398] [] ? schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:15 2020][8592379.852119] [] ? __alloc_fd+0x47/0x170 [Tue Mar 17 17:58:15 2020][8592379.857727] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:15 2020][8592379.863492] [] ? do_sys_open+0x137/0x240 [Tue Mar 17 17:58:15 2020][8592379.869260] [] ? SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:15 2020][8592379.875198] [] ? SyS_open+0x1e/0x20 [Tue Mar 17 17:58:15 2020][8592379.880514] [] ? system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:15 2020][8592379.886889] systemd-udevd S ffff9c6a30a230c0 0 5754 1 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.894161] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.896805] [] ? kmem_cache_alloc+0x35/0x1f0 [Tue Mar 17 17:58:15 2020][8592379.902920] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.908080] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:15 2020][8592379.915309] [] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0 [Tue Mar 17 17:58:15 2020][8592379.922286] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:15 2020][8592379.928815] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:15 2020][8592379.934092] [] ? do_unlinkat+0xf1/0x2d0 [Tue Mar 17 17:58:15 2020][8592379.939761] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:15 2020][8592379.945556] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:15 2020][8592379.951319] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:15 2020][8592379.957531] ccp-1-q2 S ffff9c6a3622b0c0 0 6418 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592379.964820] Call Trace: [Tue Mar 17 17:58:15 2020][8592379.967492] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:15 2020][8592379.972979] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592379.978177] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592379.984967] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592379.991722] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592379.998440] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.003536] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.009820] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.020086] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.026390] ccp-1-q3 S ffff9c6a3622e180 0 6419 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.033659] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.036338] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:15 2020][8592380.041829] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.047032] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.053827] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.060533] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.067269] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.072325] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.078634] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.085247] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.091574] ccp-1-q4 S ffff9c6a36229040 0 6420 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.098842] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.101494] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.106639] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.113480] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.120186] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.126953] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.132015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.138306] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.144940] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.151255] ccp-2-q0 S ffff9c6a2f40a080 0 6431 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.158529] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.161159] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.166303] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.173127] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.179829] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.186582] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.191642] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.197944] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.204576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.210891] ccp-2-q1 S ffff9c6a2f40e180 0 6433 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.218166] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.220813] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.225965] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.232805] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.239508] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.246255] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.251312] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.257587] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.264221] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.270504] ccp-2-q2 S ffff9c6a2f40d140 0 6434 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.277775] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.280405] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.285564] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.292419] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.299127] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.307093] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.312152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.319488] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.326120] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.332427] ccp-2-q3 S ffff9c6a2f40c100 0 6435 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.339717] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.342392] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.347540] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.354378] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.361087] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:15 2020][8592380.367853] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:15 2020][8592380.372905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.379196] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:15 2020][8592380.385825] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:15 2020][8592380.392133] ccp-2-q4 S ffff9c6a2f408000 0 6436 2 0x00000000 [Tue Mar 17 17:58:15 2020][8592380.399407] Call Trace: [Tue Mar 17 17:58:15 2020][8592380.402053] [] schedule+0x29/0x70 [Tue Mar 17 17:58:15 2020][8592380.407203] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:15 2020][8592380.414028] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.420731] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.427487] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.432543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.438862] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.445481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.451783] ccp-3-q2 S ffff9c5a39e42580 0 6476 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.459057] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.461705] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.466848] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.476555] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.483262] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.490023] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.495082] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.501401] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.508036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.514339] ccp-3-q3 S ffff9c6a33f84100 0 6477 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.521631] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.524292] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.529441] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.536289] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.542992] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.549711] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.554768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.561033] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.567661] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.573928] ccp-3-q4 S ffff9c6a33f830c0 0 6478 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.581233] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.583875] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.589061] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.595849] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.602591] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.609314] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.614392] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.620676] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.627339] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.633608] ccp-4-q0 S ffff9c6a33f86180 0 6479 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.640878] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.643507] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.648675] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.655474] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.662177] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.668899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.673953] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.680247] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.686869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.693141] ccp-4-q1 S ffff9c6a33f82080 0 6480 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.700473] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.703110] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.708305] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.715120] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.721882] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.728602] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.736295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.742581] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.749240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.755520] ccp-4-q2 S ffff9c6a33f80000 0 6481 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.762791] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.765423] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.770587] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.777384] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.784092] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.790828] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.795886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.802174] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.808790] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.815100] ccp-4-q3 S ffff9c6a33f81040 0 6482 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.822384] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.825047] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.830190] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.837015] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.843718] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.850473] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.855530] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.862476] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.869092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.875388] ccp-4-q4 S ffff9c6a39cd9040 0 6483 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.882662] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.885292] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.890439] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.897277] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.903986] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.910748] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.915806] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.922073] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.928710] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.935013] ccp-5-q2 S ffff9c6a2f409040 0 6484 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592380.942288] Call Trace: [Tue Mar 17 17:58:16 2020][8592380.944937] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592380.950085] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.956963] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592380.963666] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592380.970427] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592380.975485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.981787] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592380.988423] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592380.994728] ccp-5-q3 S ffff9c7a2e526180 0 6485 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.002002] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.006451] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.011594] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.018445] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.025147] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.031853] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.036921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.043190] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.049860] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.056144] ccp-5-q4 S ffff9c7a2e521040 0 6486 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.063451] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.066094] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.071238] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.078046] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.084774] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.091496] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.096597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.102882] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.109535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.115820] ccp-6-q0 S ffff9c7a2e522080 0 6487 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.123127] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.125773] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:16 2020][8592381.131282] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.136445] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.143242] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.149963] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.156663] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.161715] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.167997] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.174613] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.180887] ccp-6-q1 S ffff9c7a2e5230c0 0 6488 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.188215] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.190845] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:16 2020][8592381.196364] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.201521] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.208340] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.215058] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.221760] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.226834] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.233163] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.239779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.246120] ccp-6-q2 S ffff9c7a2e524100 0 6489 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.253399] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.256060] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:16 2020][8592381.261551] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.266748] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.273535] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.280283] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.286986] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.292086] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.298369] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.305013] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.311304] ccp-6-q3 S ffff9c6a3622a080 0 6490 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.318618] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.321264] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:16 2020][8592381.326796] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.331945] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.338748] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.345454] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.352172] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.358848] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.365131] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:16 2020][8592381.371791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.378079] ccp-6-q4 S ffff9c5a39eab0c0 0 6491 2 0x00000000 [Tue Mar 17 17:58:16 2020][8592381.385348] Call Trace: [Tue Mar 17 17:58:16 2020][8592381.387992] [] schedule+0x29/0x70 [Tue Mar 17 17:58:16 2020][8592381.393154] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.399962] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:16 2020][8592381.406716] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:16 2020][8592381.413437] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:16 2020][8592381.418527] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:16 2020][8592381.424810] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.431442] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.437728] ccp-7-q2 S ffff9c5a39ea8000 0 6492 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.445019] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.447662] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:17 2020][8592381.453168] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.458325] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.465137] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.471854] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.478600] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.483672] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.489970] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.496603] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.502892] ccp-7-q3 S ffff9c5a39eae180 0 6493 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.510184] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.512832] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.517982] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.524806] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.531510] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.538265] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.543321] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.549631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.556252] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.562569] ccp-7-q4 S ffff9c5a39ead140 0 6494 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.569868] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.572509] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.577735] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.584541] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.591251] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.597956] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.603016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.609284] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.615905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.622192] ccp-8-q0 S ffff9c5a39eac100 0 6495 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.629465] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.632093] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.637238] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.644043] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.650770] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.657473] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.662554] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.668840] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.675486] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.681770] ccp-8-q1 S ffff9c6a336e0000 0 6496 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.689065] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.691710] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.696876] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.703680] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.710401] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.717108] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.722203] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.728492] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.735140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.741421] ccp-8-q2 S ffff9c5ae9674100 0 6497 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.748729] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.751370] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.756533] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.763342] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.770075] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.776780] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.781874] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.788164] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.794820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.801106] ccp-8-q3 S ffff9c6a2f212080 0 6498 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.808434] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.811079] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.816236] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.823038] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.829739] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.836475] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.841534] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.847858] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.854474] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.860794] ccp-8-q4 S ffff9c6a2f214100 0 6499 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.868069] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.870717] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.875867] [] ccp_cmd_queue_thread+0x94/0x2b0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.882721] [] ? ccp_do_cmd_backlog+0xd0/0xd0 [ccp] [Tue Mar 17 17:58:17 2020][8592381.889470] [] ? ccp_unregister_rng+0x30/0x30 [ccp] [Tue Mar 17 17:58:17 2020][8592381.896197] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.901258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.907568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.914229] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.920533] kmpathd S ffff9c6a2074e180 0 11172 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.927833] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.930521] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.935668] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:17 2020][8592381.941550] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:17 2020][8592381.947738] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592381.952798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.959072] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592381.965716] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592381.971985] kmpath_handlerd S ffff9c6a20749040 0 11173 2 0x00000000 [Tue Mar 17 17:58:17 2020][8592381.979276] Call Trace: [Tue Mar 17 17:58:17 2020][8592381.981923] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592381.987082] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:17 2020][8592381.992928] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:17 2020][8592381.999128] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:17 2020][8592382.004197] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592382.010513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:17 2020][8592382.017144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:17 2020][8592382.023449] multipathd S ffff9c5a39222bc0 0 11204 1 0x00000000 [Tue Mar 17 17:58:17 2020][8592382.030744] Call Trace: [Tue Mar 17 17:58:17 2020][8592382.033401] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592382.038547] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:17 2020][8592382.044770] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:17 2020][8592382.050264] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:17 2020][8592382.055605] [] ? mntput+0x24/0x40 [Tue Mar 17 17:58:17 2020][8592382.060765] [] ? __fput+0x186/0x260 [Tue Mar 17 17:58:17 2020][8592382.066114] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:17 2020][8592382.071441] [] ? task_work_run+0xc0/0xe0 [Tue Mar 17 17:58:17 2020][8592382.077213] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:17 2020][8592382.083408] multipathd S ffff9c5a162c5140 0 11205 1 0x00000000 [Tue Mar 17 17:58:17 2020][8592382.090698] Call Trace: [Tue Mar 17 17:58:17 2020][8592382.093341] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592382.098501] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:17 2020][8592382.104695] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:17 2020][8592382.110202] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:17 2020][8592382.116656] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:17 2020][8592382.121987] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:17 2020][8592382.127315] [] ? do_futex+0x5a0/0x5a0 [Tue Mar 17 17:58:17 2020][8592382.132820] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:17 2020][8592382.139015] multipathd S ffff9c8a217f6180 0 11206 1 0x00000000 [Tue Mar 17 17:58:17 2020][8592382.146305] Call Trace: [Tue Mar 17 17:58:17 2020][8592382.148949] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592382.154107] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:17 2020][8592382.161342] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 17:58:17 2020][8592382.167033] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:17 2020][8592382.172885] [] ? __pollwait+0x7f/0xf0 [Tue Mar 17 17:58:17 2020][8592382.178398] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:17 2020][8592382.184941] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:17 2020][8592382.191230] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:17 2020][8592382.196825] [] ? netlink_recvmsg+0x212/0x490 [Tue Mar 17 17:58:17 2020][8592382.202943] [] ? sock_recvmsg+0xc5/0x100 [Tue Mar 17 17:58:17 2020][8592382.208706] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:17 2020][8592382.215167] [] ? native_smp_send_reschedule+0x52/0x70 [Tue Mar 17 17:58:17 2020][8592382.222061] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:17 2020][8592382.227566] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:17 2020][8592382.234024] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:17 2020][8592382.240046] [] ? wake_up_q+0x5b/0x80 [Tue Mar 17 17:58:17 2020][8592382.245459] [] ? futex_wake_op+0x4a5/0x610 [Tue Mar 17 17:58:17 2020][8592382.251402] [] ? do_futex+0x17a/0x5a0 [Tue Mar 17 17:58:17 2020][8592382.256910] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:17 2020][8592382.262158] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:17 2020][8592382.268358] multipathd S ffff9c8a217f5140 0 11207 1 0x00000000 [Tue Mar 17 17:58:17 2020][8592382.275646] Call Trace: [Tue Mar 17 17:58:17 2020][8592382.278292] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:17 2020][8592382.284992] [] schedule+0x29/0x70 [Tue Mar 17 17:58:17 2020][8592382.290147] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:17 2020][8592382.297299] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:17 2020][8592382.303219] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:17 2020][8592382.310554] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:17 2020][8592382.317086] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:17 2020][8592382.323351] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:17 2020][8592382.328939] [] ? get_page_from_freelist+0x412/0xa70 [Tue Mar 17 17:58:17 2020][8592382.335659] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:17 2020][8592382.341152] [] ? local_clock+0x25/0x30 [Tue Mar 17 17:58:17 2020][8592382.346761] [] ? calc_timer_values+0x1c/0x90 [Tue Mar 17 17:58:17 2020][8592382.352877] [] ? ktime_get_mono_fast_ns+0x48/0x90 [Tue Mar 17 17:58:17 2020][8592382.359426] [] ? watchdog_overflow_callback+0x3a/0x140 [Tue Mar 17 17:58:17 2020][8592382.366406] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:17 2020][8592382.371912] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:17 2020][8592382.378896] [] ? add_page_to_unevictable_list+0xc8/0x100 [Tue Mar 17 17:58:17 2020][8592382.386048] [] ? ioremap_page_range+0x2e8/0x480 [Tue Mar 17 17:58:17 2020][8592382.392423] [] ? vunmap_page_range+0x234/0x470 [Tue Mar 17 17:58:17 2020][8592382.398710] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:17 2020][8592382.404824] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:17 2020][8592382.411025] [] SyS_ppoll+0xee/0x1f0 [Tue Mar 17 17:58:17 2020][8592382.416375] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:17 2020][8592382.422576] multipathd S ffff9c5a39222bc0 0 17891 1 0x00000000 [Tue Mar 17 17:58:17 2020][8592382.429872] Call Trace: [Tue Mar 17 17:58:17 2020][8592382.432518] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.437673] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.444048] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.449280] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.455329] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.461252] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.467451] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.474430] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.480892] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.487094] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.492785] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592382.498995] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592382.505485] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592382.510732] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592382.517038] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592382.523238] multipathd S ffff9c8a35184100 0 17894 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592382.530527] Call Trace: [Tue Mar 17 17:58:18 2020][8592382.533169] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.538336] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.544706] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.549936] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.555967] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.561915] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.568101] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.575100] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.581547] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.587747] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.593434] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592382.599644] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592382.606099] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592382.611333] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592382.617607] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592382.623794] multipathd S ffff9c8a35181040 0 17895 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592382.631069] Call Trace: [Tue Mar 17 17:58:18 2020][8592382.633714] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.638862] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.645248] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.650479] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.656545] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.662475] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.668677] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.675655] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.682121] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.688324] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.694004] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592382.700204] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592382.706665] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592382.711912] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592382.718199] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592382.724392] multipathd S ffff9c8a35182080 0 17896 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592382.731693] Call Trace: [Tue Mar 17 17:58:18 2020][8592382.734322] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.739474] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.745845] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.751073] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.757104] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.763044] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.769229] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.776222] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.782664] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.788852] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.794535] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592382.800746] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592382.807204] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592382.812462] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592382.818749] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592382.824948] multipathd S ffff9c8a31bdc100 0 17898 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592382.832236] Call Trace: [Tue Mar 17 17:58:18 2020][8592382.834892] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.840049] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.846423] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.851652] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.857698] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.863625] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.869834] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.876819] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.883287] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.889485] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.895168] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592382.901367] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592382.907826] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592382.913068] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592382.919356] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592382.925555] multipathd S ffff9c8a2eb4e180 0 17899 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592382.932827] Call Trace: [Tue Mar 17 17:58:18 2020][8592382.935458] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592382.940642] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.947014] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592382.952262] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592382.958285] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.964236] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.970434] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592382.977414] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.983877] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592382.990073] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592382.995749] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592383.001931] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592383.008394] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592383.013647] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592383.019913] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592383.026121] multipathd S ffff9c6a23294100 0 17902 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592383.033396] Call Trace: [Tue Mar 17 17:58:18 2020][8592383.036042] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592383.041186] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.047576] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592383.052805] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592383.058852] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.064790] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.070987] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592383.077971] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.084432] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.090637] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592383.096303] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592383.102505] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592383.108965] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592383.114200] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592383.120517] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592383.126709] multipathd S ffff9c6a23292080 0 17903 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592383.133999] Call Trace: [Tue Mar 17 17:58:18 2020][8592383.136647] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592383.141813] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.148185] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592383.153433] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592383.159442] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.165387] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.171588] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.178033] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.184231] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592383.189913] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592383.196098] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:18 2020][8592383.201356] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592383.207800] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592383.213042] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592383.219323] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592383.225521] multipathd S ffff9c6a232930c0 0 17904 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592383.232810] Call Trace: [Tue Mar 17 17:58:18 2020][8592383.235438] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592383.240598] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.246970] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592383.252217] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592383.258241] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.264187] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.270389] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592383.277375] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.283834] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.290021] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592383.295694] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592383.301910] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592383.308353] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592383.313598] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592383.319869] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592383.326066] multipathd S ffff9c6a23291040 0 17905 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592383.333352] Call Trace: [Tue Mar 17 17:58:18 2020][8592383.335998] [] schedule+0x29/0x70 [Tue Mar 17 17:58:18 2020][8592383.341140] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.347528] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:18 2020][8592383.352760] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:18 2020][8592383.358808] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.364732] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.370941] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:18 2020][8592383.377916] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.384387] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:18 2020][8592383.390587] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:18 2020][8592383.396271] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:18 2020][8592383.402474] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:18 2020][8592383.408936] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:18 2020][8592383.414177] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:18 2020][8592383.420446] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:18 2020][8592383.426631] multipathd S ffff9c6a23295140 0 17907 1 0x00000000 [Tue Mar 17 17:58:18 2020][8592383.433938] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.436568] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.441773] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.448155] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.453394] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.459438] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.465404] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.471607] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.478609] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.485065] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.491274] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.496958] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592383.503167] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592383.509630] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592383.514877] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592383.521162] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592383.527359] multipathd S ffff9c6a23296180 0 17910 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592383.534655] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.537302] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.542457] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.548840] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.554072] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.560104] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.566033] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.572220] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.579205] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.585654] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.591857] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.597519] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592383.603706] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592383.610166] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592383.615416] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592383.621688] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592383.627896] multipathd S ffff9c6a36d10000 0 17911 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592383.635171] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.637817] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.642962] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.649351] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.654579] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.660632] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.666559] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.672762] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.679752] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.686230] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.692435] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.698122] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592383.704324] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592383.710781] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592383.716021] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592383.722308] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592383.728502] multipathd S ffff9c6a36d130c0 0 17912 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592383.735791] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.738420] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.743611] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.749967] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.755200] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.761232] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.767181] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.773366] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.780365] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.786811] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.793014] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.798694] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592383.804894] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592383.811355] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592383.816605] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592383.822895] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592383.829102] multipathd S ffff9c5a39222bc0 0 17913 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592383.836394] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.839042] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.844196] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.850581] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.855813] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.861844] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.867773] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.873994] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.880968] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.887436] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.893645] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.899328] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592383.905531] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592383.911988] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592383.917230] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592383.923516] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592383.929718] multipathd S ffff9c6a36d14100 0 17917 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592383.937008] Call Trace: [Tue Mar 17 17:58:19 2020][8592383.939650] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592383.944812] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.951185] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592383.956433] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592383.962464] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.968417] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.974596] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592383.981574] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.988016] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592383.994219] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592383.999893] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592384.006076] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592384.012537] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592384.017785] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592384.024059] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592384.030275] multipathd S ffff9c6a36d16180 0 17919 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592384.037550] Call Trace: [Tue Mar 17 17:58:19 2020][8592384.040197] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592384.045346] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.051746] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592384.056973] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592384.063019] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.068958] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.075139] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592384.082117] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.088585] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.094770] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592384.100471] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592384.106655] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:19 2020][8592384.111886] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592384.118340] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592384.123589] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592384.129863] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592384.136081] multipathd S ffff9c7a3268c100 0 17921 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592384.143353] Call Trace: [Tue Mar 17 17:58:19 2020][8592384.146000] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592384.151152] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.157540] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592384.162769] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592384.168809] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.174732] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.180935] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592384.187920] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.194380] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.200579] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592384.206257] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592384.212452] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592384.218915] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592384.224161] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592384.230449] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592384.236649] multipathd S ffff9c7a32688000 0 17922 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592384.243939] Call Trace: [Tue Mar 17 17:58:19 2020][8592384.246569] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592384.251733] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.258099] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592384.263347] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592384.269378] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.275318] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.281518] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:19 2020][8592384.288499] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.294962] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.301161] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592384.306854] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592384.313042] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:19 2020][8592384.319494] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:19 2020][8592384.324742] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:19 2020][8592384.331029] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:19 2020][8592384.337212] multipathd S ffff9c7a3268e180 0 17923 1 0x00000000 [Tue Mar 17 17:58:19 2020][8592384.344497] Call Trace: [Tue Mar 17 17:58:19 2020][8592384.347130] [] schedule+0x29/0x70 [Tue Mar 17 17:58:19 2020][8592384.352278] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.358677] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:19 2020][8592384.363908] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:19 2020][8592384.369957] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.375888] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.382075] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.388539] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:19 2020][8592384.394725] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:19 2020][8592384.400412] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:19 2020][8592384.406600] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:19 2020][8592384.411853] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.418311] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.423558] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.429827] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.436030] multipathd S ffff9c7a3268d140 0 17924 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.443318] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.445962] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.451106] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.457477] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.462708] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.468756] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.474677] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.480894] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.487862] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.496872] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.503072] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592384.508759] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592384.514955] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.521438] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.526683] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.532969] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.539171] multipathd S ffff9c7a3268a080 0 17925 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.546458] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.549115] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.554284] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.560653] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.565900] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.571927] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.577873] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.584074] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.591057] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.597518] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.603716] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592384.609382] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592384.615583] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.622042] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.627288] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.633568] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.639780] multipathd S ffff9c7a32689040 0 17926 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.647070] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.649715] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.654859] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.661264] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.666497] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.672544] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.678469] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.684670] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.691638] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.698090] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.704291] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592384.709968] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592384.716162] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.722624] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.727872] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.734157] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.740352] multipathd S ffff9c7a2e332080 0 17927 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.747641] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.750283] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.755429] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.761799] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.767047] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.773070] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.778995] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.785184] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.792168] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.798636] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.804832] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592384.810508] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592384.816710] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.823183] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.828433] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.834714] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.840914] multipathd S ffff9c7a2e335140 0 17930 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.848199] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.850845] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.855989] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.862379] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.867608] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.873649] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.879572] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.885755] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.892733] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.899198] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.905394] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592384.911069] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592384.917271] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592384.923733] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592384.928990] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592384.935274] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592384.941478] multipathd S ffff9c7a2e334100 0 17931 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592384.948768] Call Trace: [Tue Mar 17 17:58:20 2020][8592384.951397] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592384.956561] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.962931] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592384.968178] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592384.974208] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.980150] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.986346] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592384.993315] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592384.999796] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.005983] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592385.011665] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592385.017847] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592385.024327] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592385.029555] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592385.035846] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592385.042041] multipathd S ffff9c7a20e84100 0 17932 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592385.049328] Call Trace: [Tue Mar 17 17:58:20 2020][8592385.051973] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592385.057135] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.063507] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592385.068757] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592385.074786] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.080741] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.086941] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592385.093929] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.100388] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.106586] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592385.112266] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592385.118466] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592385.124920] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592385.130167] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592385.136460] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592385.142664] multipathd S ffff9c7a20e80000 0 17933 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592385.149950] Call Trace: [Tue Mar 17 17:58:20 2020][8592385.152579] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592385.157739] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.164112] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592385.169356] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592385.175381] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.181320] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.187522] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.193982] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.200180] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592385.205879] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592385.212072] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:20 2020][8592385.217319] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592385.223772] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592385.229002] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592385.235281] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592385.241464] multipathd S ffff9c7a3288c100 0 17934 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592385.248750] Call Trace: [Tue Mar 17 17:58:20 2020][8592385.251392] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592385.256540] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.262928] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592385.268157] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592385.274198] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.280122] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.286323] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592385.293300] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.299762] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.305958] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592385.311641] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592385.317827] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:20 2020][8592385.324284] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:20 2020][8592385.329532] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:20 2020][8592385.335818] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:20 2020][8592385.342010] multipathd S ffff9c7a3288b0c0 0 17935 1 0x00000000 [Tue Mar 17 17:58:20 2020][8592385.349302] Call Trace: [Tue Mar 17 17:58:20 2020][8592385.351946] [] schedule+0x29/0x70 [Tue Mar 17 17:58:20 2020][8592385.357091] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.363463] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:20 2020][8592385.368710] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:20 2020][8592385.374727] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.380655] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.386853] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:20 2020][8592385.393837] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.400285] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:20 2020][8592385.406485] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:20 2020][8592385.412148] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:20 2020][8592385.418349] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.424810] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.430043] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.436342] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.442524] multipathd S ffff9c7a39b64100 0 17936 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.449809] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.452438] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.457612] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.463965] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.469209] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.475234] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.481189] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.487374] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592385.494376] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.500832] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.507031] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592385.512719] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592385.518906] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.525369] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.530642] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.536929] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.543131] multipathd S ffff9c7a39b62080 0 17937 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.550430] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.553082] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.558237] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.564613] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.569851] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.575900] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.581830] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.588050] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592385.595021] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.601499] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.607686] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592385.613401] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592385.619585] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.626026] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.631274] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.637577] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.643756] multipathd S ffff9c7a37352080 0 17938 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.651052] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.653681] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.658848] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.665220] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.670469] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.676500] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.682449] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.688649] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592385.695632] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.702092] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.708290] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592385.713958] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592385.720176] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.726639] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.731890] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.738180] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.744376] multipathd S ffff9c7a37350000 0 17940 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.751662] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.754310] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.759454] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.765826] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.771053] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.777093] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.783030] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.789255] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.795720] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.801925] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592385.807597] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592385.813815] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:21 2020][8592385.819053] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.825498] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.830738] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.837037] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.843222] multipathd S ffff9c7a37351040 0 17941 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.850515] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.853145] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.858311] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.864685] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.869931] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.875957] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.881905] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.888102] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592385.895090] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.901568] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.907790] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592385.913473] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592385.919670] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592385.926122] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592385.931376] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592385.937661] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592385.943859] multipathd S ffff9c7a37356180 0 17942 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592385.951128] Call Trace: [Tue Mar 17 17:58:21 2020][8592385.953778] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592385.958937] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.965325] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592385.970554] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592385.976602] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.982525] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592385.988727] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592385.995710] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.002174] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.008374] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592386.014072] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592386.020257] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592386.026724] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592386.031974] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592386.038270] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592386.044471] multipathd S ffff9c7a37355140 0 17943 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592386.051757] Call Trace: [Tue Mar 17 17:58:21 2020][8592386.054404] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592386.059580] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.065944] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592386.071219] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592386.077280] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.083229] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.089427] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592386.096416] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.102878] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.109091] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592386.114763] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592386.120974] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592386.127420] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592386.132646] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592386.138932] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592386.145118] multipathd S ffff9c7a37354100 0 17944 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592386.152405] Call Trace: [Tue Mar 17 17:58:21 2020][8592386.155049] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592386.160194] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.166585] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592386.171812] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592386.177850] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.183776] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.189979] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592386.196962] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.203423] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.209622] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592386.215301] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592386.221494] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592386.227955] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592386.233203] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592386.239489] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592386.245683] multipathd S ffff9c7a2f61b0c0 0 17945 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592386.252974] Call Trace: [Tue Mar 17 17:58:21 2020][8592386.255604] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592386.260790] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.267164] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592386.272417] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592386.278449] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.284395] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.290597] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:21 2020][8592386.297585] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.304052] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.310252] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592386.315928] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592386.322126] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:21 2020][8592386.328579] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:21 2020][8592386.333828] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:21 2020][8592386.340118] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:21 2020][8592386.346316] multipathd S ffff9c7a2f61a080 0 17946 1 0x00000000 [Tue Mar 17 17:58:21 2020][8592386.353603] Call Trace: [Tue Mar 17 17:58:21 2020][8592386.356235] [] schedule+0x29/0x70 [Tue Mar 17 17:58:21 2020][8592386.361387] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.367784] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:21 2020][8592386.373021] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:21 2020][8592386.379070] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.384997] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.391203] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.397666] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:21 2020][8592386.403865] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:21 2020][8592386.409545] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:21 2020][8592386.415745] [] ? up_read+0x1f/0x30 [Tue Mar 17 17:58:21 2020][8592386.420986] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592386.427444] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592386.432684] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592386.438976] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.445176] multipathd S ffff9c7a2f618000 0 17947 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.452472] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.455116] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.460280] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.466653] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592386.471900] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592386.477928] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.483852] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.490053] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592386.497035] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.503495] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.509691] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592386.515367] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592386.521566] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592386.528020] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592386.533268] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592386.539558] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.545756] multipathd S ffff9c7a2f619040 0 17948 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.553041] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.555702] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:22 2020][8592386.562404] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.567583] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:22 2020][8592386.573074] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:22 2020][8592386.579105] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:22 2020][8592386.585046] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:22 2020][8592386.590727] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:22 2020][8592386.596323] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.602520] multipathd S ffff9c7a2f61d140 0 17949 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.609809] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.612454] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.617596] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:22 2020][8592386.623826] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:22 2020][8592386.629321] [] ? mntput+0x24/0x40 [Tue Mar 17 17:58:22 2020][8592386.634486] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:22 2020][8592386.640427] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:22 2020][8592386.645761] [] ? lockref_put_or_lock+0x48/0x80 [Tue Mar 17 17:58:22 2020][8592386.652048] [] ? mntput+0x24/0x40 [Tue Mar 17 17:58:22 2020][8592386.657215] [] ? __fput+0x186/0x260 [Tue Mar 17 17:58:22 2020][8592386.662543] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:22 2020][8592386.667886] [] ? task_work_run+0xc0/0xe0 [Tue Mar 17 17:58:22 2020][8592386.673656] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.679866] multipathd S ffff9c6a22791040 0 17982 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.687157] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.689805] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.694957] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.701329] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592386.706557] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592386.714371] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.720316] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.726534] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592386.733522] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.739972] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.746172] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592386.751854] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592386.758062] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592386.764525] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592386.769769] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592386.776056] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.782272] multipathd S ffff9c6a22790000 0 18000 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.789565] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.792206] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.797370] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.803745] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592386.808992] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592386.815025] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.820973] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.827171] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592386.834157] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.840632] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.846839] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592386.852521] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592386.858701] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592386.865154] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592386.870402] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592386.876680] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.882900] multipathd S ffff9c6a22795140 0 18004 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.890168] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.892814] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.897969] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.904343] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592386.909577] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592386.915625] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.921557] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.927759] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592386.934742] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.941202] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592386.947404] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592386.953081] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592386.959288] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592386.965766] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592386.971009] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592386.977299] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592386.983499] multipathd S ffff9c6a227930c0 0 18010 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592386.990788] Call Trace: [Tue Mar 17 17:58:22 2020][8592386.993417] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592386.998582] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.004953] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592387.010189] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592387.016233] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.022159] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.028364] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592387.035347] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.041809] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.048008] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592387.053700] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592387.059896] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592387.066355] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592387.071596] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592387.077884] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592387.084086] multipathd S ffff9c8a31bdb0c0 0 18015 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592387.091374] Call Trace: [Tue Mar 17 17:58:22 2020][8592387.094015] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592387.099181] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.105538] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592387.110785] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592387.116814] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.122752] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.128951] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592387.135937] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.142385] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.148586] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592387.154260] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592387.160458] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592387.166912] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592387.172160] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592387.178443] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592387.184624] multipathd S ffff9c5a39222bc0 0 18021 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592387.191917] Call Trace: [Tue Mar 17 17:58:22 2020][8592387.194577] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592387.199724] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.206097] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592387.211324] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592387.217365] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.223289] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.229491] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592387.236475] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.242964] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.249161] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592387.254848] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592387.261028] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592387.267469] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592387.272715] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592387.279004] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592387.285191] multipathd S ffff9c5a39222bc0 0 18031 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592387.292494] Call Trace: [Tue Mar 17 17:58:22 2020][8592387.295124] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592387.300293] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.306665] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592387.311914] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592387.317929] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.323872] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.330073] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:22 2020][8592387.337056] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.343518] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.349714] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:22 2020][8592387.355387] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:22 2020][8592387.361591] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:22 2020][8592387.368036] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:22 2020][8592387.373280] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:22 2020][8592387.379560] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:22 2020][8592387.385759] multipathd S ffff9c5a39222bc0 0 18036 1 0x00000000 [Tue Mar 17 17:58:22 2020][8592387.393046] Call Trace: [Tue Mar 17 17:58:22 2020][8592387.395694] [] schedule+0x29/0x70 [Tue Mar 17 17:58:22 2020][8592387.400842] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.407231] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:22 2020][8592387.412465] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:22 2020][8592387.418528] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:22 2020][8592387.424456] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.430646] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.437619] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.444085] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.450268] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.455948] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.462134] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.468594] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.473839] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.480105] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.486318] multipathd S ffff9c8a2ebf0000 0 18064 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.493605] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.496236] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592387.501390] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.507780] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592387.513017] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592387.519047] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.524977] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.531162] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.538155] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.544598] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.550811] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.556490] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.562700] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.569163] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.574409] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.580686] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.586889] multipathd S ffff9c8a2ebf1040 0 18069 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.594175] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.596820] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592387.601965] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.608337] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592387.613566] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592387.619597] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.625519] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.631724] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.638708] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.645176] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.651373] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.657054] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.663247] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.669707] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.674949] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.681234] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.687428] multipathd S ffff9c8a2ebf2080 0 18136 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.694719] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.697363] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592387.702530] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.708906] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592387.714154] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592387.720185] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.726134] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.732318] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.739305] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.745766] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.751963] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.757645] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.763828] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.770282] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.775529] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.781796] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.788013] multipathd S ffff9c8a2ebf30c0 0 18153 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.795285] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.797933] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592387.803088] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.809475] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592387.814704] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592387.820751] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.826676] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.832906] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.839889] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.846365] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.852567] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.858250] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.864447] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.870888] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.876129] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.882417] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.888603] multipathd S ffff9c8a2ebf4100 0 18171 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.895932] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.898565] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592387.903728] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.910090] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592387.915323] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592387.921354] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.927296] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.933480] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592387.940482] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.946925] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592387.953127] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592387.958799] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592387.965000] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592387.971453] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592387.976700] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592387.982980] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592387.989179] multipathd S ffff9c8a2ebf5140 0 18182 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592387.996466] Call Trace: [Tue Mar 17 17:58:23 2020][8592387.999097] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592388.004263] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.010634] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592388.015883] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592388.021906] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.027847] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.034045] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592388.041027] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.047491] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.053694] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592388.059369] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592388.065570] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592388.072030] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592388.077261] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592388.083548] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592388.089751] multipathd S ffff9c8a2ebf6180 0 18194 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592388.097027] Call Trace: [Tue Mar 17 17:58:23 2020][8592388.099669] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592388.104817] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.111188] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592388.116419] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592388.122467] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.128397] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.134608] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592388.141585] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.148053] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.154253] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592388.159931] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592388.166127] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592388.172586] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592388.177825] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592388.184114] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592388.190313] multipathd S ffff9c8a21bea080 0 18224 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592388.197613] Call Trace: [Tue Mar 17 17:58:23 2020][8592388.200257] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592388.205424] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.211793] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592388.217039] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592388.223071] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.229016] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.235216] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592388.242199] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.248663] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.254868] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592388.260541] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592388.266749] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592388.273188] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592388.278448] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592388.284715] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592388.290914] multipathd S ffff9c8a21bec100 0 18234 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592388.298208] Call Trace: [Tue Mar 17 17:58:23 2020][8592388.300852] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592388.305999] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.312387] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:23 2020][8592388.317615] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:23 2020][8592388.323659] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.329587] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.335788] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:23 2020][8592388.342765] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.349226] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:23 2020][8592388.355427] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:23 2020][8592388.361104] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:23 2020][8592388.367299] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:23 2020][8592388.373775] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:23 2020][8592388.379017] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:23 2020][8592388.385301] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:23 2020][8592388.391495] multipathd S ffff9c8a21be9040 0 18245 1 0x00000000 [Tue Mar 17 17:58:23 2020][8592388.398785] Call Trace: [Tue Mar 17 17:58:23 2020][8592388.401413] [] schedule+0x29/0x70 [Tue Mar 17 17:58:23 2020][8592388.406587] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.412942] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.418192] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.424214] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.437220] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.443433] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.450400] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.456879] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.463061] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.468759] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.474941] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.481400] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.486649] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.492934] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592388.499125] multipathd S ffff9c8a21bee180 0 18269 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592388.506417] Call Trace: [Tue Mar 17 17:58:24 2020][8592388.509060] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592388.514223] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.520594] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.525843] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.531872] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.537795] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.543996] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.550979] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.557442] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.563662] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.569337] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.575552] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.582006] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.587254] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.593535] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592388.599735] multipathd S ffff9c8a21beb0c0 0 18278 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592388.607028] Call Trace: [Tue Mar 17 17:58:24 2020][8592388.609674] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592388.614819] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.621175] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.626402] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.632449] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.638374] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.644576] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.651553] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.658014] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.664212] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.669890] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.676085] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.682544] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.687798] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.694086] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592388.700282] multipathd S ffff9c8a21bed140 0 18286 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592388.707571] Call Trace: [Tue Mar 17 17:58:24 2020][8592388.710214] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592388.715379] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.721737] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.726971] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.733002] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.738959] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.745148] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.752136] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.758581] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.764794] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.770460] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.776661] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.783120] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.788365] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.794648] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592388.800845] multipathd S ffff9c8a1efae180 0 18294 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592388.808140] Call Trace: [Tue Mar 17 17:58:24 2020][8592388.810783] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592388.815930] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.822317] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.827548] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.833575] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.839515] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.845696] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.852682] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.859152] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.865336] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.871016] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.877201] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.883666] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.888914] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.895200] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592388.901393] multipathd S ffff9c8a1efab0c0 0 18298 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592388.908692] Call Trace: [Tue Mar 17 17:58:24 2020][8592388.911333] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592388.916510] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.922892] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592388.928152] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592388.934183] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.940124] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.946320] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592388.953307] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.959766] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592388.965964] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592388.971637] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592388.977836] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592388.984305] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592388.989556] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592388.995843] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592389.002043] multipathd S ffff9c8a1efa8000 0 18302 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592389.009331] Call Trace: [Tue Mar 17 17:58:24 2020][8592389.011975] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592389.017123] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.023511] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592389.028738] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592389.034755] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.040682] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.046886] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592389.053863] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.060331] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.066530] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592389.072210] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592389.078413] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592389.084872] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592389.090112] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592389.096398] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592389.102606] multipathd S ffff9c8a1efa9040 0 18321 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592389.109880] Call Trace: [Tue Mar 17 17:58:24 2020][8592389.112509] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592389.117703] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.124065] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592389.129317] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592389.135348] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.141297] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.147497] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592389.154477] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.160926] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.167111] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592389.172793] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592389.178993] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592389.185440] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592389.190696] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592389.196969] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592389.203172] multipathd S ffff9c8a1efac100 0 18325 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592389.210457] Call Trace: [Tue Mar 17 17:58:24 2020][8592389.213104] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592389.218249] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.224654] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592389.229884] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592389.235933] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.241864] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.248067] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592389.255053] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.261519] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.267720] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592389.273400] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592389.279599] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592389.286061] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592389.291308] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592389.297598] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592389.303797] multipathd S ffff9c8a1efad140 0 18331 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592389.311071] Call Trace: [Tue Mar 17 17:58:24 2020][8592389.313715] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592389.318881] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.325239] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:24 2020][8592389.330473] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:24 2020][8592389.336563] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.342494] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.348681] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:24 2020][8592389.355671] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.362133] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:24 2020][8592389.368319] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:24 2020][8592389.374016] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:24 2020][8592389.380198] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:24 2020][8592389.386659] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:24 2020][8592389.391890] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:24 2020][8592389.398190] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:24 2020][8592389.404369] multipathd S ffff9c5a39222bc0 0 18337 1 0x00000000 [Tue Mar 17 17:58:24 2020][8592389.411658] Call Trace: [Tue Mar 17 17:58:24 2020][8592389.414301] [] schedule+0x29/0x70 [Tue Mar 17 17:58:24 2020][8592389.419448] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.425803] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.431033] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.437061] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.442984] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.449168] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.456132] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.462580] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.468766] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.474433] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.480611] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.487051] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.492277] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592389.498554] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592389.504755] multipathd S ffff9c8a1c631040 0 18344 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592389.512052] Call Trace: [Tue Mar 17 17:58:25 2020][8592389.514682] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592389.519834] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.526206] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.531435] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.537464] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.543387] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.549573] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.556559] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.563008] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.569194] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.574881] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.581082] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.587524] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.592764] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592389.599035] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592389.605222] multipathd S ffff9c5a39222bc0 0 18385 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592389.612506] Call Trace: [Tue Mar 17 17:58:25 2020][8592389.615137] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592389.620288] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.626677] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.631907] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.637923] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.643896] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.650082] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.657086] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.663535] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.669759] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.675438] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.681653] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.688120] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.693353] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592389.699634] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592389.705833] multipathd S ffff9c5a39222bc0 0 18391 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592389.713104] Call Trace: [Tue Mar 17 17:58:25 2020][8592389.715746] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592389.720892] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.727283] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.732511] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.738572] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.744514] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.750700] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.757664] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.764137] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.770337] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.776001] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.782242] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.788690] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.793964] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592389.800246] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592389.806430] multipathd S ffff9c5a39222bc0 0 18404 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592389.813732] Call Trace: [Tue Mar 17 17:58:25 2020][8592389.816376] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592389.821521] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.827894] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.833122] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.839145] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.845116] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.851296] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.858290] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.864747] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.870927] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.876600] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.882785] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.889246] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.894473] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592389.900756] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592389.906938] multipathd S ffff9c5a39222bc0 0 18414 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592389.914215] Call Trace: [Tue Mar 17 17:58:25 2020][8592389.916861] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592389.922013] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.928385] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592389.933615] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592389.939646] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.945621] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.951821] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592389.958805] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.965263] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592389.971487] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592389.977169] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592389.983375] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592389.989846] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592389.995120] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592390.001402] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592390.007621] multipathd S ffff9c5a39222bc0 0 18423 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592390.014912] Call Trace: [Tue Mar 17 17:58:25 2020][8592390.017541] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592390.022748] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.029103] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592390.034358] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592390.040402] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.046328] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.052525] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592390.059538] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.065998] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.072218] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592390.077894] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592390.084108] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592390.090563] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592390.095806] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592390.102106] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592390.108291] multipathd S ffff9c8a36920000 0 18434 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592390.115610] Call Trace: [Tue Mar 17 17:58:25 2020][8592390.118240] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592390.123407] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.129768] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592390.135002] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592390.141020] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.146947] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.153131] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592390.160095] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.166544] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.172744] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592390.178406] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592390.184608] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592390.191051] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592390.196337] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592390.202613] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592390.208797] multipathd S ffff9c5a39222bc0 0 18443 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592390.216085] Call Trace: [Tue Mar 17 17:58:25 2020][8592390.218716] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592390.223863] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.230252] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592390.235484] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592390.241500] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.247445] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.253631] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592390.260609] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.267060] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.273245] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592390.278930] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592390.285150] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592390.291592] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592390.296872] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592390.303145] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592390.309325] multipathd S ffff9c8a36922080 0 18490 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592390.316630] Call Trace: [Tue Mar 17 17:58:25 2020][8592390.319256] [] schedule+0x29/0x70 [Tue Mar 17 17:58:25 2020][8592390.324399] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.330757] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:25 2020][8592390.335982] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:25 2020][8592390.342020] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.347946] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.354129] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:25 2020][8592390.361106] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.367551] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:25 2020][8592390.373736] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:25 2020][8592390.379418] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:25 2020][8592390.385600] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:25 2020][8592390.392070] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:25 2020][8592390.397326] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:25 2020][8592390.403590] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:25 2020][8592390.409787] multipathd S ffff9c5a39222bc0 0 18495 1 0x00000000 [Tue Mar 17 17:58:25 2020][8592390.417057] Call Trace: [Tue Mar 17 17:58:25 2020][8592390.419688] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.424838] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.431210] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.436438] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.442489] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.448419] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.454625] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.461618] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.468066] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.474264] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.479927] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.486122] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.492564] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592390.497790] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592390.504072] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592390.510254] multipathd S ffff9c8a36924100 0 18508 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592390.517522] Call Trace: [Tue Mar 17 17:58:26 2020][8592390.520176] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.525319] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.531691] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.536919] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.542944] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.548866] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.555053] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.562037] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.568478] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.574664] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.580337] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.586522] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.592982] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592390.598211] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592390.604479] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592390.610678] multipathd S ffff9c8a36926180 0 18519 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592390.617951] Call Trace: [Tue Mar 17 17:58:26 2020][8592390.620582] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.625731] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.632133] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.637358] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.643369] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.649309] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.655491] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.662475] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.668917] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.675104] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.680787] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.686966] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.693407] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592390.698666] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592390.704935] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592390.711147] multipathd S ffff9c8a34600000 0 18526 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592390.718433] Call Trace: [Tue Mar 17 17:58:26 2020][8592390.721104] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.726250] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.732620] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.737849] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.743891] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.749827] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.756006] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.762983] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.769424] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.775610] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.781284] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.787467] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.793907] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592390.799156] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592390.805422] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592390.811606] multipathd S ffff9c8a34601040 0 18536 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592390.818923] Call Trace: [Tue Mar 17 17:58:26 2020][8592390.821554] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.826705] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.833093] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.838325] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.844369] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.850309] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.856488] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.863466] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.869921] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.876104] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.881786] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.887967] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.894445] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592390.899676] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592390.905965] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592390.912146] multipathd S ffff9c8a34602080 0 18544 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592390.919414] Call Trace: [Tue Mar 17 17:58:26 2020][8592390.922059] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592390.927206] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.933591] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592390.938823] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592390.944846] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.950769] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.956952] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592390.963932] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.970381] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592390.976566] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592390.982239] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592390.988438] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592390.994878] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592391.000119] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592391.006390] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592391.012578] multipathd S ffff9c8a346030c0 0 18556 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592391.019870] Call Trace: [Tue Mar 17 17:58:26 2020][8592391.022500] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592391.027661] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.034033] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592391.039262] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592391.045298] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.051223] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.057411] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592391.064408] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.070857] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.077041] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592391.082736] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592391.088924] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592391.095389] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592391.100637] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592391.106906] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592391.113107] multipathd S ffff9c8a34604100 0 18563 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592391.120379] Call Trace: [Tue Mar 17 17:58:26 2020][8592391.123010] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592391.128163] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.134535] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592391.139761] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592391.145796] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.151726] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.157909] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592391.164898] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.171345] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.177531] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592391.183222] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592391.189404] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592391.195844] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592391.201093] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592391.207361] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592391.213556] multipathd S ffff9c8a34605140 0 18575 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592391.220826] Call Trace: [Tue Mar 17 17:58:26 2020][8592391.223458] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592391.228609] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.234979] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592391.240209] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592391.246240] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.252164] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.258347] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592391.265323] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.271768] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.277951] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592391.283638] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592391.289824] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592391.296280] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592391.301523] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592391.307792] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592391.313992] multipathd S ffff9c8a34606180 0 18587 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592391.321280] Call Trace: [Tue Mar 17 17:58:26 2020][8592391.323913] [] schedule+0x29/0x70 [Tue Mar 17 17:58:26 2020][8592391.329063] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.335435] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:26 2020][8592391.340663] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:26 2020][8592391.346702] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.352631] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.358849] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:26 2020][8592391.365836] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.372303] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:26 2020][8592391.378503] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:26 2020][8592391.384164] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:26 2020][8592391.390365] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:26 2020][8592391.396806] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:26 2020][8592391.402034] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:26 2020][8592391.408330] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:26 2020][8592391.414515] multipathd S ffff9c8a3ecad140 0 18597 1 0x00000000 [Tue Mar 17 17:58:26 2020][8592391.421791] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.424421] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.429587] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.435946] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.441171] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.447198] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.453126] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.459313] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.466298] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.472747] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.478934] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.484618] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.490797] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.497267] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592391.502513] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592391.508801] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592391.515000] multipathd S ffff9c5a39222bc0 0 18667 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592391.522289] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.524946] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.530096] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.536470] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.541698] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.547728] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.553656] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.559838] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.566821] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.573267] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.579478] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.585143] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.591337] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.597780] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592391.603005] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592391.609286] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592391.615468] multipathd S ffff9c5a39222bc0 0 18683 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592391.622737] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.625379] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.630525] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.636911] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.642136] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.648159] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.654081] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.660267] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.667243] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.673686] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.679872] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.685553] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.691737] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.698184] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592391.703439] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592391.709712] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592391.715899] multipathd S ffff9c8a3eca8000 0 18696 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592391.723186] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.725812] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.730957] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.737330] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.742557] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.748581] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.754516] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.760697] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.767674] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.774118] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.780303] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.785983] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.792166] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.798605] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592391.803845] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592391.810114] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592391.816294] multipathd S ffff9c8a3eca9040 0 18702 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592391.823588] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.826218] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.831360] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.837731] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.842961] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.848982] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.854910] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.861093] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.868078] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.874533] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.880716] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.886397] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.892578] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.899032] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592391.904261] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592391.910527] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592391.916721] multipathd S ffff9c8a37576180 0 18816 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592391.923991] Call Trace: [Tue Mar 17 17:58:27 2020][8592391.926621] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592391.931762] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.938150] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592391.943385] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592391.949400] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.955344] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.961532] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592391.968494] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.974957] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592391.981146] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592391.986826] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592391.993023] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592391.999467] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592392.004715] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592392.010984] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592392.017171] multipathd S ffff9c5a39222bc0 0 19050 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592392.024458] Call Trace: [Tue Mar 17 17:58:27 2020][8592392.027132] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592392.032284] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.038679] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592392.043909] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592392.049945] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.055872] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.062056] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592392.069032] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.075476] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.081663] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592392.087357] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592392.093542] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592392.099996] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592392.105242] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592392.111510] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592392.117722] multipathd S ffff9c5a39222bc0 0 19078 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592392.124989] Call Trace: [Tue Mar 17 17:58:27 2020][8592392.127620] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592392.132763] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.139136] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592392.144365] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592392.150395] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.156329] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.162514] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592392.169511] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.175958] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.182142] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592392.187819] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592392.193999] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592392.200439] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592392.205687] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592392.211960] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592392.218158] multipathd S ffff9c8a35180000 0 19148 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592392.225431] Call Trace: [Tue Mar 17 17:58:27 2020][8592392.228061] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592392.233211] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.239634] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592392.244867] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592392.250884] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.256837] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.263020] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592392.269998] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.276449] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.282633] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592392.288321] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592392.294507] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592392.300960] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592392.306188] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592392.312457] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592392.318660] multipathd S ffff9c8a37572080 0 19162 1 0x00000000 [Tue Mar 17 17:58:27 2020][8592392.325928] Call Trace: [Tue Mar 17 17:58:27 2020][8592392.328558] [] schedule+0x29/0x70 [Tue Mar 17 17:58:27 2020][8592392.333699] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.340074] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:27 2020][8592392.345317] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:27 2020][8592392.351330] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.357302] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.363484] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:27 2020][8592392.370450] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.376910] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:27 2020][8592392.383105] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:27 2020][8592392.388780] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:27 2020][8592392.394962] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:27 2020][8592392.401416] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:27 2020][8592392.406665] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:27 2020][8592392.412938] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:27 2020][8592392.419130] multipathd S ffff9c5a39222bc0 0 19263 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.426403] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.429033] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.434184] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.440556] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.445784] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.451806] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.457731] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.463915] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.470905] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.477351] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.483536] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.489209] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.495393] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592392.501832] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592392.507071] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592392.513341] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592392.519521] multipathd S ffff9c5a39222bc0 0 19377 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.526814] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.529444] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.534596] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.540970] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.546198] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.552206] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.558149] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.564327] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.571292] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.577784] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.583968] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.589669] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.595848] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592392.602288] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592392.607530] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592392.613800] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592392.619983] multipathd S ffff9c8a3550a080 0 19399 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.627264] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.629892] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.635035] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.641420] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.646658] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.652683] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.658608] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.664793] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.671768] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.678212] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.684397] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.690079] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.696262] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592392.702703] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592392.707963] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592392.714237] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592392.720430] multipathd S ffff9c8a3550e180 0 19615 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.727719] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.730348] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.735489] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.741864] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.747093] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.753122] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.759073] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.765257] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.772233] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.778690] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.784871] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.790553] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.796736] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592392.803182] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592392.808424] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592392.814690] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592392.820871] multipathd S ffff9c5a39222bc0 0 19665 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.828166] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.830796] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.835945] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.842319] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.847547] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.853570] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.859508] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.865686] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.872667] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.879128] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.885307] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.890982] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.897164] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592392.903603] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592392.908843] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592392.915114] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592392.921295] multipathd S ffff9c8a2fd0d140 0 19995 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592392.928586] Call Trace: [Tue Mar 17 17:58:28 2020][8592392.931218] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592392.936368] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.942756] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592392.947987] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592392.954010] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.959932] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.966118] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592392.973102] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.979559] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592392.985741] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592392.991422] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592392.997618] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592393.004059] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592393.009299] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592393.015570] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592393.021750] multipathd S ffff9c8a2fd0a080 0 20040 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592393.029043] Call Trace: [Tue Mar 17 17:58:28 2020][8592393.031673] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592393.036817] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.043187] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592393.048415] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592393.054455] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.060379] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.066565] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592393.073542] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.079984] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.086168] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592393.091858] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592393.098043] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592393.104492] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592393.109752] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592393.116039] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592393.122224] multipathd S ffff9c8a2fd0b0c0 0 20098 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592393.129516] Call Trace: [Tue Mar 17 17:58:28 2020][8592393.132146] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592393.137288] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.143666] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592393.148897] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592393.154921] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.160860] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.167047] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592393.174036] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.180483] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.186669] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592393.192342] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592393.198524] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592393.204979] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592393.210238] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592393.216507] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592393.222699] multipathd S ffff9c5a39222bc0 0 20673 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592393.229986] Call Trace: [Tue Mar 17 17:58:28 2020][8592393.232621] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592393.237770] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.244154] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592393.249380] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592393.255402] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.261327] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.267512] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592393.274494] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.280941] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.287126] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592393.292821] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592393.299007] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592393.305455] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592393.310724] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592393.316991] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592393.323193] multipathd S ffff9c5a39222bc0 0 21032 1 0x00000000 [Tue Mar 17 17:58:28 2020][8592393.330480] Call Trace: [Tue Mar 17 17:58:28 2020][8592393.333113] [] schedule+0x29/0x70 [Tue Mar 17 17:58:28 2020][8592393.338263] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.344635] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:28 2020][8592393.349863] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:28 2020][8592393.355927] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.361850] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.368037] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:28 2020][8592393.375015] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.381467] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:28 2020][8592393.387653] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:28 2020][8592393.393330] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:28 2020][8592393.399516] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:28 2020][8592393.405962] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:28 2020][8592393.411216] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:28 2020][8592393.417490] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:28 2020][8592393.423677] multipathd S ffff9c8a2bfea080 0 24529 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.430963] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.433592] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.438735] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.445115] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.450345] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.456378] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.462308] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.468493] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.475483] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.481928] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.488115] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.493797] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592393.499980] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592393.506428] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592393.511678] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592393.517944] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592393.524124] multipathd S ffff9c5a39222bc0 0 25217 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.531419] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.534049] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.539200] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.545572] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.550801] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.556825] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.562754] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.568940] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.575918] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.582368] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.588554] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.594250] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592393.600433] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592393.606890] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592393.612120] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592393.618394] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592393.624592] multipathd S ffff9c8a2bfe6180 0 25235 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.631865] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.634496] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.639648] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.646033] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.651265] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.657287] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.663211] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.669395] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.676402] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.682849] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.689035] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.694708] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592393.700892] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592393.707340] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592393.712602] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592393.718874] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592393.725067] multipathd S ffff9c5a39222bc0 0 25251 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.732339] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.734970] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.740122] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.746493] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.751721] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.757746] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.763691] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.769868] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.776864] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.783314] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.789500] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.795175] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592393.801358] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592393.807803] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592393.813046] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592393.819315] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592393.825502] multipathd S ffff9c8a33e3c100 0 25383 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.832805] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.835433] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.840577] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.846950] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.852178] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.858210] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.864141] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.870328] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.877311] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.883767] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.889948] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.895629] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592393.901813] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592393.908262] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592393.913509] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592393.919780] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592393.925966] multipathd S ffff9c5a39222bc0 0 26041 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592393.933251] Call Trace: [Tue Mar 17 17:58:29 2020][8592393.935881] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592393.941022] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.947410] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592393.952643] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592393.958677] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.964606] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.970790] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592393.977767] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.984213] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592393.990396] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592393.996092] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592394.002277] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592394.008743] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592394.013978] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592394.020249] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592394.026444] multipathd S ffff9c5a39222bc0 0 26347 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592394.033715] Call Trace: [Tue Mar 17 17:58:29 2020][8592394.036344] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592394.041488] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.047860] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592394.053088] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592394.059129] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.065052] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.071236] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592394.078213] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.084664] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.090866] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592394.096544] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592394.102738] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592394.109197] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592394.114425] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592394.120728] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592394.126912] multipathd S ffff9c8a1f7d9040 0 27527 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592394.134181] Call Trace: [Tue Mar 17 17:58:29 2020][8592394.136823] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592394.141972] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.148341] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592394.153571] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592394.159595] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.165519] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.171700] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592394.178681] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.185133] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.191314] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592394.196988] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592394.203171] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592394.209610] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592394.214852] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592394.221139] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592394.227325] multipathd S ffff9c5a39222bc0 0 27778 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592394.234612] Call Trace: [Tue Mar 17 17:58:29 2020][8592394.237240] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592394.242385] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.248785] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592394.254010] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592394.260031] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.265955] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.272141] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592394.279120] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.285569] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.291756] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592394.297449] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592394.303638] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592394.310098] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592394.315327] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592394.321595] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592394.327794] multipathd S ffff9c8a30bb4100 0 27840 1 0x00000000 [Tue Mar 17 17:58:29 2020][8592394.335068] Call Trace: [Tue Mar 17 17:58:29 2020][8592394.337695] [] schedule+0x29/0x70 [Tue Mar 17 17:58:29 2020][8592394.342840] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.349212] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:29 2020][8592394.354441] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:29 2020][8592394.360500] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.366429] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.372615] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:29 2020][8592394.379593] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.386044] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:29 2020][8592394.392228] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:29 2020][8592394.397911] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:29 2020][8592394.404092] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:29 2020][8592394.410540] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:29 2020][8592394.415797] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:29 2020][8592394.422068] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:29 2020][8592394.428255] multipathd S ffff9c5a39222bc0 0 30895 1 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.435541] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.438170] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.443321] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.449695] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:30 2020][8592394.454924] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:30 2020][8592394.460956] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.466884] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.473071] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:30 2020][8592394.480062] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.486506] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.492695] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:30 2020][8592394.498375] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:30 2020][8592394.504558] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:30 2020][8592394.511018] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:30 2020][8592394.516286] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:30 2020][8592394.522559] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:30 2020][8592394.528776] multipathd S ffff9c5a39222bc0 0 31227 1 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.536066] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.538696] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.543847] [] dm_wait_event+0xa1/0xf0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.550221] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:30 2020][8592394.555448] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:30 2020][8592394.561471] [] dev_wait+0x50/0xd0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.567394] [] ctl_ioctl+0x24e/0x550 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.573595] [] ? __mem_cgroup_commit_charge+0xe2/0x2f0 [Tue Mar 17 17:58:30 2020][8592394.580574] [] ? table_deps+0x1a0/0x1a0 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.587025] [] dm_ctl_ioctl+0xe/0x20 [dm_mod] [Tue Mar 17 17:58:30 2020][8592394.593239] [] do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 17:58:30 2020][8592394.598903] [] ? __set_task_blocked+0x41/0xa0 [Tue Mar 17 17:58:30 2020][8592394.605099] [] ? __set_current_blocked+0x42/0x70 [Tue Mar 17 17:58:30 2020][8592394.611537] [] SyS_ioctl+0xa1/0xc0 [Tue Mar 17 17:58:30 2020][8592394.616763] [] ? SyS_rt_sigprocmask+0xc4/0x100 [Tue Mar 17 17:58:30 2020][8592394.623046] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:30 2020][8592394.629229] kdmflush S ffff9c6a2373a080 0 12448 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.636513] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.639156] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.644301] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592394.650169] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592394.656355] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.661415] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.667694] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.674313] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.680585] bioset S ffff9c6a2373b0c0 0 12454 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.687876] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.690510] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.695647] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592394.701499] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592394.707695] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.712764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.719046] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.725679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.731959] kdmwork-253:0 S ffff9c8a34ab4100 0 12455 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.739232] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.741875] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.747017] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:30 2020][8592394.753039] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:30 2020][8592394.758699] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.763752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.770048] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.776668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.782943] kdmflush S ffff9c8a34ab30c0 0 12456 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.790241] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.792890] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.798031] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592394.803888] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592394.810068] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.815129] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.821423] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.828044] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.834316] bioset S ffff9c8a34ab0000 0 12457 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.841607] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.844242] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.849388] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592394.855235] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592394.861417] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.866470] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.872752] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.879380] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.885650] kdmwork-253:1 S ffff9c8a34ab5140 0 12458 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.892935] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.895564] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.900703] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:30 2020][8592394.906725] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:30 2020][8592394.912387] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.917449] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.923737] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.930353] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.936620] kdmflush S ffff9c8a34ab2080 0 12473 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.943926] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.946559] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592394.951699] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592394.957546] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592394.963730] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592394.968794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.975079] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592394.981695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592394.987969] bioset S ffff9c8a34ab6180 0 12474 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592394.995268] Call Trace: [Tue Mar 17 17:58:30 2020][8592394.997901] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.003042] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.008898] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.015079] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.020132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.026421] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.033036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.039304] kdmwork-253:2 S ffff9c7a1f3c0000 0 12475 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.046605] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.049236] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.054383] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:30 2020][8592395.060418] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:30 2020][8592395.066081] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.071135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.077414] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.084031] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.090298] kdmflush S ffff9c7a1f3c4100 0 12482 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.097612] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.100237] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.105392] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.111240] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.117424] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.122492] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.128762] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.135383] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.141673] bioset S ffff9c7a1f3c5140 0 12483 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.148945] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.151571] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.156711] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.162571] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.168757] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.173809] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.180091] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.186705] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.192972] kdmwork-253:3 S ffff9c7a1f3c6180 0 12484 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.200258] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.202890] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.208034] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:30 2020][8592395.214054] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:30 2020][8592395.219730] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.224789] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.231077] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.237691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.243959] kdmflush S ffff9c7a1f3c1040 0 12485 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.251248] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.253874] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.259022] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.264877] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.271059] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.276145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.282440] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.289077] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.295356] bioset S ffff9c7a1f3c2080 0 12486 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.302633] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.305281] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.310423] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.316286] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.322470] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.327525] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.333852] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.340472] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.346768] kdmwork-253:4 S ffff9c6a2430c100 0 12488 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.354057] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.356685] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.361827] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:30 2020][8592395.367855] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:30 2020][8592395.373518] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.378579] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.384857] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:30 2020][8592395.391489] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:30 2020][8592395.397758] kdmflush S ffff9c6a2430d140 0 12512 2 0x00000000 [Tue Mar 17 17:58:30 2020][8592395.405050] Call Trace: [Tue Mar 17 17:58:30 2020][8592395.407682] [] schedule+0x29/0x70 [Tue Mar 17 17:58:30 2020][8592395.412820] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:30 2020][8592395.418675] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:30 2020][8592395.424860] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:30 2020][8592395.429922] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.436208] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.442826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.449098] bioset S ffff9c6a24308000 0 12513 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.456398] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.459030] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.464172] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.470029] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.476209] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.481262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.487546] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.494167] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.500431] kdmwork-253:5 S ffff9c6a2430b0c0 0 12514 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.507729] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.510354] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.515494] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592395.521515] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592395.527179] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.532231] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.538513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.545128] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.551403] kdmflush S ffff9c6a2430a080 0 12515 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.558700] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.561335] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.566482] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.572366] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.578547] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.583608] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.589896] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.596511] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.602778] bioset S ffff9c6a2430e180 0 12516 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.610067] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.612692] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.617830] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.623683] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.629870] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.634938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.641221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.647838] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.654124] kdmwork-253:6 S ffff9c6a24309040 0 12517 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.661412] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.664045] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.669194] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592395.675234] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592395.680899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.685969] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.692236] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.698851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.705152] kdmflush S ffff9c6a2539a080 0 12521 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.712430] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.715065] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.720206] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.726061] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.732256] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.737312] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.743601] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.750217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.756492] bioset S ffff9c6a2539c100 0 12522 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.763791] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.766422] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.771562] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.777411] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.783591] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.788646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.794945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.801558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.807824] kdmwork-253:7 S ffff9c6a2539d140 0 12523 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.815111] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.817739] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.822878] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592395.828898] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592395.834561] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.839614] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.845906] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.852519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.858793] kdmflush S ffff9c6a2539e180 0 12566 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.866091] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.868719] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.873865] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.879711] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.885893] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.890946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.897241] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.903876] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.910145] bioset S ffff9c6a25398000 0 12568 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.917440] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.920068] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.925216] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592395.931074] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592395.937253] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.942306] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.948600] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592395.955219] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.961485] kdmwork-253:8 S ffff9c6a25399040 0 12569 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592395.968780] Call Trace: [Tue Mar 17 17:58:31 2020][8592395.971409] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592395.976556] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592395.982576] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592395.988238] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592395.993292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592395.999570] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.006190] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.012480] kdmflush S ffff9c6a2539b0c0 0 12589 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.019779] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.022413] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.027562] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.033417] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.039614] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.044671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.050957] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.057573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.063841] bioset S ffff9c8a3a31a080 0 12590 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.071138] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.073773] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.078936] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.084796] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.090976] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.096067] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.102338] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.108960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.115247] kdmwork-253:9 S ffff9c8a3a31e180 0 12591 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.122535] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.125168] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.130315] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592396.136342] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592396.142019] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.147076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.153363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.159982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.166254] kdmflush S ffff9c8a3a31d140 0 12597 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.173546] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.176178] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.181324] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.187186] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.193372] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.198425] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.204713] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.211331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.217605] bioset S ffff9c8a2e7ba080 0 12598 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.224921] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.227552] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.232695] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.238550] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.244746] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.249828] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.256103] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.262737] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.269025] kdmwork-253:10 S ffff9c8a2e7bd140 0 12599 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.276303] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.278944] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.284088] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:31 2020][8592396.290130] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:31 2020][8592396.295796] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.300849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.307135] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.313752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.320019] kdmflush S ffff9c8a18e9c100 0 12600 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.327308] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.329934] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.335075] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.340929] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.347112] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.352165] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.358467] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.365085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.371352] bioset S ffff9c8a18e9d140 0 12601 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.378641] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.381267] [] schedule+0x29/0x70 [Tue Mar 17 17:58:31 2020][8592396.386406] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:31 2020][8592396.392253] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:31 2020][8592396.398437] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:31 2020][8592396.403497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.409778] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:31 2020][8592396.416408] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:31 2020][8592396.422677] kdmwork-253:11 S ffff9c8a18e9a080 0 12602 2 0x00000000 [Tue Mar 17 17:58:31 2020][8592396.429963] Call Trace: [Tue Mar 17 17:58:31 2020][8592396.432593] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.437741] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592396.443769] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592396.449430] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.454500] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.460781] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.467399] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.473699] kdmflush S ffff9c8a18e99040 0 12604 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.480981] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.483628] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.488768] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.494616] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.500801] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.505877] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.512158] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.518774] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.525063] bioset S ffff9c8a18e98000 0 12605 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.532373] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.535007] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.540154] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.546002] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.552184] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.557246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.563535] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.570148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.576429] kdmwork-253:12 S ffff9c5a17bd5140 0 12606 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.583703] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.586328] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.591471] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592396.597512] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592396.603193] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.608247] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.614530] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.621159] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.627427] kdmflush S ffff9c5a17bd1040 0 12607 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.634717] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.637358] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.642500] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.648370] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.654554] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.659608] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.665888] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.672519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.678786] bioset S ffff9c5a17bd2080 0 12610 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.686077] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.688709] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.693851] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.699720] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.705905] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.710958] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.717247] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.723862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.730127] kdmwork-253:13 S ffff9c5a17bd30c0 0 12611 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.737413] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.740043] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.745183] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592396.751203] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592396.756882] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.761935] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.768217] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.774833] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.781108] kdmflush S ffff9c5a17bd6180 0 12654 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.788402] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.791030] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.796169] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.802017] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.808197] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.813251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.819544] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.826157] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.832431] bioset S ffff9c5a17bd0000 0 12655 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.839721] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.842354] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.847494] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.853348] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.859530] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.864598] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.870881] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.877499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.883787] kdmwork-253:14 S ffff9c8a37bd4100 0 12656 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.891056] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.893687] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.898824] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592396.904848] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592396.910512] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.915569] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.921850] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.928480] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.934752] kdmflush S ffff9c8a37bd0000 0 12657 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.942047] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.944675] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592396.949821] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592396.955674] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592396.961858] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592396.966913] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.973201] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592396.979820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592396.986092] bioset S ffff9c8a37bd6180 0 12658 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592396.993400] Call Trace: [Tue Mar 17 17:58:32 2020][8592396.996033] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.001173] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.007027] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.013208] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.018261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.024542] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.031159] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.037425] kdmwork-253:15 S ffff9c8a37bd5140 0 12660 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.044720] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.047349] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.052487] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592397.058508] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592397.064169] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.069223] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.075504] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.082118] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.088386] kdmflush S ffff9c8a37bd30c0 0 12663 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.095698] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.098327] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.103473] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.109333] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.115521] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.120573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.126852] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.133469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.139736] bioset S ffff9c8a37bd2080 0 12664 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.147024] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.149651] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.154797] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.160649] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.166836] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.171890] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.178179] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.184794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.191060] kdmwork-253:16 S ffff9c8a37bd1040 0 12665 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.198345] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.200975] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.206124] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592397.212165] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592397.217831] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.222883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.229165] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.235780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.242047] kdmflush S ffff9c8a2f626180 0 12666 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.249337] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.251968] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.257110] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.262964] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.269161] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.274217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.280498] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.287112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.293378] bioset S ffff9c8a2f620000 0 12667 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.300671] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.303303] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.308450] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.314298] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.320481] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.325542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.331836] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.338456] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.344730] kdmwork-253:17 S ffff9c8a2f622080 0 12668 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.352016] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.354690] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.359834] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:32 2020][8592397.365874] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:32 2020][8592397.371537] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.376598] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.382899] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:32 2020][8592397.389520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.395786] kdmflush S ffff9c8a2f6230c0 0 12669 2 0x00000000 [Tue Mar 17 17:58:32 2020][8592397.403084] Call Trace: [Tue Mar 17 17:58:32 2020][8592397.405719] [] schedule+0x29/0x70 [Tue Mar 17 17:58:32 2020][8592397.410866] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:32 2020][8592397.416722] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:32 2020][8592397.422903] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:32 2020][8592397.427956] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:32 2020][8592397.434237] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.440866] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.447136] bioset S ffff9c8a2f621040 0 12670 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.454426] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.457059] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.462197] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.468047] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.474228] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.479289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.485569] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.492185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.498450] kdmwork-253:18 S ffff9c8a2f625140 0 12671 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.505747] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.508374] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.513513] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592397.519534] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592397.525197] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.530258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.536536] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.543153] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.549420] kdmflush S ffff9c8a2f624100 0 12674 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.556719] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.559353] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.564491] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.570339] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.576520] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.581575] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.587862] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.594481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.600753] bioset S ffff9c8a3242d140 0 12675 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.608044] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.610675] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.615816] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.621682] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.627863] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.632917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.639205] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.645821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.652108] kdmwork-253:19 S ffff9c8a3242b0c0 0 12676 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.659382] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.662009] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.667150] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592397.673162] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592397.678824] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.683899] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.690168] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.696779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.703059] kdmflush S ffff9c8a3242c100 0 12677 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.710335] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.712961] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.718099] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.723948] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.730130] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.735184] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.741471] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.748087] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.754353] bioset S ffff9c8a3242a080 0 12678 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.761643] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.764269] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.769417] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.775259] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.781460] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.786531] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.792798] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.799426] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.805695] kdmwork-253:20 S ffff9c8a32429040 0 12679 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.812972] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.815601] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.820753] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592397.826763] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592397.832422] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.837504] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.843777] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.850390] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.856671] kdmflush S ffff9c8a32428000 0 12680 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.863944] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.866569] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.871708] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.877542] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.883723] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.888798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.895081] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.901695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.907975] bioset S ffff9c8a323b30c0 0 12681 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.915253] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.917888] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.923034] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592397.928881] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592397.935063] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.940116] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.946398] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592397.953027] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.959296] kdmwork-253:21 S ffff9c8a323b4100 0 12682 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592397.966583] Call Trace: [Tue Mar 17 17:58:33 2020][8592397.969211] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592397.974358] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592397.980378] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592397.986041] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592397.991094] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592397.997374] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.003989] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.010257] kdmflush S ffff9c8a323b1040 0 12683 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.017564] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.020196] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.025337] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.031184] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.037366] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.042417] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.048698] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.055315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.061579] bioset S ffff9c8a323b2080 0 12684 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.068880] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.071512] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.076652] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.082484] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.088664] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.093739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.100023] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.106640] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.112926] kdmwork-253:22 S ffff9c8a323b5140 0 12685 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.120216] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.122845] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.127985] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592398.134005] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592398.139674] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.144728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.151008] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.157624] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.163893] kdmflush S ffff9c8a323b0000 0 12686 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.171189] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.173815] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.178961] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.184810] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.190993] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.196054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.202335] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.208949] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.215213] bioset S ffff9c8a393a2080 0 12687 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.222504] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.225130] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.230269] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.236132] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.242315] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.247370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.253657] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.260274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.266547] kdmwork-253:23 S ffff9c8a393a4100 0 12688 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.273835] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.276465] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.281612] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:33 2020][8592398.287640] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:33 2020][8592398.293315] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.298374] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.304662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.311279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.317551] kdmflush S ffff9c8a393a30c0 0 12689 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.324833] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.327467] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.332607] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.338462] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.344645] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.349705] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.356016] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.362635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.368921] bioset S ffff9c8a393a6180 0 12690 2 0x00000000 [Tue Mar 17 17:58:33 2020][8592398.376218] Call Trace: [Tue Mar 17 17:58:33 2020][8592398.378851] [] schedule+0x29/0x70 [Tue Mar 17 17:58:33 2020][8592398.383993] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:33 2020][8592398.389831] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:33 2020][8592398.396011] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:33 2020][8592398.401078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:33 2020][8592398.407363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:33 2020][8592398.413979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.420268] kdmwork-253:24 S ffff9c8a393a5140 0 12691 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.427538] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.430169] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.435316] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592398.441322] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592398.446979] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.452049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.458316] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.464930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.471225] kdmflush S ffff9c8a3e8a6180 0 12692 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.478501] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.481129] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.486278] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.492119] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.498305] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.503371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.509641] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.516255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.522555] bioset S ffff9c8a3e8a5140 0 12693 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.529828] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.532461] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.537602] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.543441] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.549621] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.554690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.560956] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.567570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.573850] kdmwork-253:25 S ffff9c8a3e8a30c0 0 12694 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.581138] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.583768] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.588907] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592398.594912] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592398.600573] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.605639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.611908] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.618522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.624802] kdmflush S ffff9c8a3e8a4100 0 12695 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.632091] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.634722] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.639866] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.645701] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.651879] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.656946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.663216] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.669829] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.676119] bioset S ffff9c8a3e8a2080 0 12696 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.683393] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.686042] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.691185] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.697039] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.703220] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.708274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.714555] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.721171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.727445] kdmwork-253:26 S ffff9c8a3e8a1040 0 12697 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.734732] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.737360] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.742499] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592398.748519] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592398.754181] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.759236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.765523] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.772139] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.778404] kdmflush S ffff9c8a1eeb5140 0 12698 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.785693] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.788319] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.793458] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.799295] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.805495] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.810552] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.816838] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.823455] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.829721] bioset S ffff9c8a1eeb4100 0 12699 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.837011] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.839644] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.844782] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.850616] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.856795] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.861876] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.868150] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.874769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.881049] kdmwork-253:27 S ffff9c8a1eeb30c0 0 12700 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.888322] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.890951] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.896090] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592398.902097] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592398.907757] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.912824] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.919107] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.925723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.932003] kdmflush S ffff9c8a1eeb1040 0 12701 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.939280] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.941910] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.947051] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592398.952883] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592398.959063] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592398.964131] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.970401] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592398.977021] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592398.983300] bioset S ffff9c8a1eeb0000 0 12702 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592398.990578] Call Trace: [Tue Mar 17 17:58:34 2020][8592398.993210] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592398.998349] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.004182] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592399.010362] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.015426] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.021699] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.028319] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.034613] kdmwork-253:28 S ffff9c8a1eeb2080 0 12703 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.041888] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.044520] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.049665] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592399.055676] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592399.061341] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.066416] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.072686] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.079304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.085584] kdmflush S ffff9c8a1f275140 0 12704 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.092879] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.095513] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.100679] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.106520] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592399.112721] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.117791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.124061] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.130694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.136966] bioset S ffff9c8a1f271040 0 12705 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.144244] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.146886] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.152027] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.157888] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592399.164075] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.169135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.175416] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.182032] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.188298] kdmwork-253:29 S ffff9c8a1f276180 0 12706 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.195585] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.198212] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.203353] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592399.209380] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592399.215043] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.220095] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.226378] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.232993] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.239257] kdmflush S ffff9c8a1f274100 0 12707 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.246547] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.249173] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.254313] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.260168] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592399.266350] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.271403] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.277689] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.284309] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.290584] bioset S ffff9c8a1f2730c0 0 12708 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.297880] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.300507] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.305654] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.311488] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:34 2020][8592399.317694] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.322769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.329038] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.335680] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.341950] kdmwork-253:30 S ffff9c8a1f270000 0 12709 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.349220] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.351861] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.357025] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:34 2020][8592399.363037] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:34 2020][8592399.368705] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:34 2020][8592399.373812] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.380085] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:34 2020][8592399.386705] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:34 2020][8592399.392984] kdmflush S ffff9c8a2eb4c100 0 12710 2 0x00000000 [Tue Mar 17 17:58:34 2020][8592399.400262] Call Trace: [Tue Mar 17 17:58:34 2020][8592399.402895] [] schedule+0x29/0x70 [Tue Mar 17 17:58:34 2020][8592399.408036] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:34 2020][8592399.413877] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.420065] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.425140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.431422] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.438040] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.444327] bioset S ffff9c8a2eb4a080 0 12711 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.451603] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.454237] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.459376] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.465209] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.471398] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.476472] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.482742] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.489361] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.495658] kdmwork-253:31 S ffff9c8a2eb4b0c0 0 12712 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.502933] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.505562] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.510710] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592399.516725] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592399.522391] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.527457] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.533727] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.540342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.546644] kdmflush S ffff9c8a2eb48000 0 12713 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.553924] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.556557] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.561695] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.567527] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.573708] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.578786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.585053] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.591665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.597945] bioset S ffff9c8a2eb4d140 0 12714 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.605238] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.607873] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.613019] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.618866] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.625049] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.630103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.636390] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.643007] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.649272] kdmwork-253:32 S ffff9c8a2eb49040 0 12715 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.656558] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.659190] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.664334] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592399.670356] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592399.676018] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.681070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.687354] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.693966] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.700233] kdmflush S ffff9c8a2e8b6180 0 12716 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.707523] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.710156] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.715295] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.721143] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.727323] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.732379] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.738667] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.745283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.751549] bioset S ffff9c8a2e8b0000 0 12717 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.758839] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.761471] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.766611] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.772459] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.778641] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.783693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.789974] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.796589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.802855] kdmwork-253:33 S ffff9c8a2e8b30c0 0 12718 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.810142] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.812771] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.817916] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592399.823925] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592399.829606] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.834681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.840955] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.847582] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.853849] kdmflush S ffff9c8a2e8b2080 0 12719 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.861142] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.863776] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.868921] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.874755] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.880935] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.886022] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.892290] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.898909] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.905201] bioset S ffff9c8a2e8b4100 0 12720 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.912473] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.915109] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.920256] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592399.926097] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592399.932275] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.937344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.943626] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592399.950241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.956522] kdmwork-253:34 S ffff9c8a1ef19040 0 12721 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592399.963794] Call Trace: [Tue Mar 17 17:58:35 2020][8592399.966423] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592399.971564] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592399.977576] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592399.983236] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592399.988305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592399.994574] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.001193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.007487] kdmflush S ffff9c8a1ef1b0c0 0 12722 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.014768] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.017400] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.022539] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.028375] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.034560] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.039631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.045906] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.052519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.058819] bioset S ffff9c8a1ef1a080 0 12723 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.066092] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.068726] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.073874] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.079715] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.085904] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.090976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.097248] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.103885] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.110164] kdmwork-253:35 S ffff9c8a1ef1e180 0 12724 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.117436] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.120094] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.125255] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592400.131279] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592400.136940] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.142006] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.148275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.154891] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.161178] kdmflush S ffff9c8a1ef1d140 0 12725 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.168454] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.171089] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.176237] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.182093] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.188273] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.193327] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.199616] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.206230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.212498] bioset S ffff9c8a1ef1c100 0 12726 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.219789] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.222420] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.227559] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.233407] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.239590] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.244650] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.250931] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.257547] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.263814] kdmwork-253:36 S ffff9c8a38f0e180 0 12727 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.271099] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.273728] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.278867] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:35 2020][8592400.284886] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:35 2020][8592400.290550] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.295604] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.301895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.308517] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.314804] kdmflush S ffff9c8a38f0c100 0 12729 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.322081] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.324714] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.329854] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.335686] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.341880] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.346950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.353219] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.359847] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.366115] bioset S ffff9c8a38f09040 0 12730 2 0x00000000 [Tue Mar 17 17:58:35 2020][8592400.373388] Call Trace: [Tue Mar 17 17:58:35 2020][8592400.376034] [] schedule+0x29/0x70 [Tue Mar 17 17:58:35 2020][8592400.381179] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:35 2020][8592400.387018] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:35 2020][8592400.393199] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:35 2020][8592400.398276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.404543] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:35 2020][8592400.411155] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:35 2020][8592400.417435] kdmwork-253:37 S ffff9c8a38f0d140 0 12731 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.424708] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.427338] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.432484] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:36 2020][8592400.438491] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:36 2020][8592400.444150] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.449217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.455501] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.462116] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.468397] kdmflush S ffff9c8a38f08000 0 12732 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.475678] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.478304] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.483446] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.489300] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.495484] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.500546] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.506835] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.513466] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.519732] bioset S ffff9c8a38f0b0c0 0 12733 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.527020] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.529646] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.534787] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.540634] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.546817] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.551878] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.558157] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.564775] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.571049] kdmwork-253:38 S ffff9c8a2fb6c100 0 12734 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.578342] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.580972] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.586110] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:36 2020][8592400.592131] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:36 2020][8592400.597795] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.602857] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.609143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.615760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.622026] kdmflush S ffff9c8a2fb6d140 0 12754 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.629322] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.631949] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.637088] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.642937] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.649119] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.654172] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.660461] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.667077] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.673351] bioset S ffff9c8a2fb6a080 0 12755 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.680646] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.683275] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.688422] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.694291] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.700478] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.705531] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.711818] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.718434] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.724702] kdmwork-253:39 S ffff9c8a2fb69040 0 12756 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.731994] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.734624] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.739765] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:36 2020][8592400.745779] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:36 2020][8592400.751447] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.756523] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.762792] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.769412] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.775696] kdmflush S ffff9c8a2fb6e180 0 13412 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.782968] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.785602] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.790742] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.796599] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.802780] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.807842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.814131] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.820747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.827021] bioset S ffff9c8a2fb6b0c0 0 13413 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.834318] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.836952] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.842092] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.847945] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:36 2020][8592400.854143] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.859200] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.865493] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.872111] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.878378] kdmwork-253:40 S ffff9c8a2ee2d140 0 13414 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.885663] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.888293] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.893433] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:36 2020][8592400.899453] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:36 2020][8592400.905116] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.910169] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.916448] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.923066] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.929338] kworker/44:1H S ffff9c5a16e1a080 0 14170 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.936629] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.939264] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592400.945451] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592400.950591] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592400.956345] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592400.963058] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592400.968129] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.974399] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592400.981033] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592400.987301] kworker/37:1H S ffff9c6a362d9040 0 14203 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592400.994575] Call Trace: [Tue Mar 17 17:58:36 2020][8592400.997224] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.003405] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.008546] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.014300] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.021013] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.026085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.032351] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.038978] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.045249] kworker/36:1H S ffff9c5a16e18000 0 14204 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.052528] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.055168] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.061349] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.066489] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.072235] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.078948] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.084021] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.090289] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.096917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.103200] kworker/32:1H S ffff9c5a16e1d140 0 14221 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.110474] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.113114] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.119296] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.124451] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.130199] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.136899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.141979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.148252] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.154865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.161145] kworker/28:1H S ffff9c5a16e1b0c0 0 14237 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.168420] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.171046] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.177226] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.182388] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.188137] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.194837] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.199925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.206199] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.212819] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.219100] kworker/29:1H S ffff9c6a24fc5140 0 14238 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.226376] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.229009] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.235189] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.240351] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.246099] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.252798] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.257882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.264154] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.270776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.277063] kworker/20:1H S ffff9c5a2976d140 0 14273 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.284339] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.286987] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.293171] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.298315] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.304061] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.310776] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.315847] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.322116] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.328743] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.335012] kworker/21:1H S ffff9c6a24fc6180 0 14274 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.342285] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.344931] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.351115] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.356266] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:36 2020][8592401.362019] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:36 2020][8592401.368741] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:36 2020][8592401.373810] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.380078] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:36 2020][8592401.386706] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:36 2020][8592401.392977] kworker/17:1H S ffff9c6a24fc4100 0 14294 2 0x00000000 [Tue Mar 17 17:58:36 2020][8592401.400259] Call Trace: [Tue Mar 17 17:58:36 2020][8592401.402905] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:36 2020][8592401.409087] [] schedule+0x29/0x70 [Tue Mar 17 17:58:36 2020][8592401.414235] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.419988] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:37 2020][8592401.426701] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.431773] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.438041] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.444671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.450938] kworker/9:1H S ffff9c6a24fc2080 0 14306 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.458211] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.460859] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:37 2020][8592401.467042] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.472189] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.477937] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:37 2020][8592401.484657] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.489728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.495996] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.502623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.508893] kworker/4:1H S ffff9c5a277d2080 0 14341 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.516165] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.518813] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:37 2020][8592401.524993] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.530138] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.535889] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:37 2020][8592401.542589] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.547679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.553949] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.560565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.566854] kworker/1:1H S ffff9c6a24fc30c0 0 14342 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.574128] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.576775] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:37 2020][8592401.582960] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.588105] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.593854] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:37 2020][8592401.600576] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.605644] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.611913] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.618540] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.624809] kdmflush S ffff9c8a2e1dd140 0 17976 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.632083] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.634731] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.639871] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.645707] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592401.651896] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.656969] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.663253] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.669867] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.676148] bioset S ffff9c8a2e1de180 0 17978 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.683423] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.686050] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.691195] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.697043] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592401.703226] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.708279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.714561] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.721187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.727459] kdmwork-253:41 S ffff9c8a2e1da080 0 17980 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.734754] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.737379] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.742520] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:37 2020][8592401.748542] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:37 2020][8592401.754204] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.759256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.765537] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.772154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.778427] kdmflush S ffff9c8a2e1db0c0 0 17997 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.785728] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.788360] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.793498] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.799346] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592401.805529] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.810604] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.816875] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.823496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.829781] bioset S ffff9c8a2e1d9040 0 17998 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.837075] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.839703] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.844850] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.850704] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592401.856886] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.861940] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.868221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.874835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.881102] kdmwork-253:42 S ffff9c8a2e1dc100 0 17999 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.888388] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.891016] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.896156] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:37 2020][8592401.902192] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:37 2020][8592401.907856] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.912910] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.919199] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.925816] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.932088] kdmflush S ffff9c5a2bf7c100 0 18001 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.939378] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.942012] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.947152] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592401.953007] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592401.959188] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592401.964241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.970523] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592401.977137] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592401.983402] bioset S ffff9c5a2bf7e180 0 18002 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592401.990694] Call Trace: [Tue Mar 17 17:58:37 2020][8592401.993329] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592401.998473] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.004322] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592402.010505] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.015558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.021837] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.028454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.034721] kdmwork-253:43 S ffff9c5a2bf7d140 0 18003 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.042013] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.044643] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.049782] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:37 2020][8592402.055805] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:37 2020][8592402.061481] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.066537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.072825] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.079440] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.085705] kdmflush S ffff9c5a2bf79040 0 18007 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.092997] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.095630] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.100769] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.106630] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592402.112815] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.117884] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.124181] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.130800] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.137088] bioset S ffff9c5a2bf7a080 0 18008 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.144364] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.146997] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.152135] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.157972] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592402.164158] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.169209] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.175502] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.182115] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.188391] kdmwork-253:44 S ffff9c8a217f30c0 0 18009 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.195683] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.198313] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.203451] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:37 2020][8592402.209459] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:37 2020][8592402.215117] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.220184] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.226454] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.233076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.239378] kworker/45:1H S ffff9c6a341fa080 0 18011 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.246656] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.249280] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:37 2020][8592402.255461] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.260602] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.266356] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:37 2020][8592402.273076] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.278134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.284399] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.291038] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.297304] kdmflush S ffff9c8a217f4100 0 18012 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.304578] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.307224] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.312366] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.318200] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592402.324387] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.329454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.335724] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.342336] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.348629] bioset S ffff9c8a35148000 0 18013 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.355902] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.358556] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.363700] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:37 2020][8592402.369532] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:37 2020][8592402.375710] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:37 2020][8592402.380778] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.387048] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:37 2020][8592402.393661] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:37 2020][8592402.399938] kdmwork-253:45 S ffff9c8a35149040 0 18014 2 0x00000000 [Tue Mar 17 17:58:37 2020][8592402.407229] Call Trace: [Tue Mar 17 17:58:37 2020][8592402.409859] [] schedule+0x29/0x70 [Tue Mar 17 17:58:37 2020][8592402.414999] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592402.421015] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592402.426687] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.431756] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.438027] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.444648] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.450929] kworker/41:1H S ffff9c6a20748000 0 18016 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.458204] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.460837] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:38 2020][8592402.467026] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.472198] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.477945] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:38 2020][8592402.484645] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.489720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.495990] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.502610] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.508891] kdmflush S ffff9c6a2074b0c0 0 18018 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.516167] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.518799] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.523939] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.529773] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.535960] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.541026] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.547297] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.553917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.560196] bioset S ffff9c6a2074a080 0 18019 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.567474] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.570107] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.575256] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.581096] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.587274] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.592327] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.598609] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.605224] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.611491] kdmwork-253:46 S ffff9c6a2074c100 0 18020 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.618785] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.621412] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.626551] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592402.632573] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592402.638234] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.643289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.649577] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.656192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.662461] kworker/33:1H S ffff9c6a24f14100 0 18026 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.669759] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.672391] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:38 2020][8592402.678571] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.683720] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.689487] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:38 2020][8592402.696205] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.701261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.707541] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.714158] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.720432] kdmflush S ffff9c6a24f16180 0 18028 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.727720] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.730346] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.735485] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.741317] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.747511] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.752583] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.758851] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.765477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.771746] bioset S ffff9c6a24f15140 0 18029 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.779020] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.781654] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.786814] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.792652] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.798839] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.803919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.810192] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.816804] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.823085] kdmwork-253:47 S ffff9c6a24f11040 0 18030 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.830357] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.832986] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.838125] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592402.844131] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592402.849789] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.854842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.861137] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.867758] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.874032] kdmflush S ffff9c6a24f12080 0 18033 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.881320] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.883946] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.889085] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.894919] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.901097] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.906165] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.912434] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.919056] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.925356] bioset S ffff9c8a3514e180 0 18034 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.932629] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.935263] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.940410] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592402.946242] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592402.952422] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592402.957488] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.963757] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592402.970371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592402.976665] kdmwork-253:48 S ffff9c8a3514d140 0 18035 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592402.983942] Call Trace: [Tue Mar 17 17:58:38 2020][8592402.986569] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592402.991707] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592402.997716] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592403.003381] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.008448] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.014718] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.021331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.027610] kworker/25:1H S ffff9c6a37fe5140 0 18047 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.034905] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.037539] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:38 2020][8592403.043727] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.048873] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.054621] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:38 2020][8592403.061343] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.066397] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.072665] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.079299] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.085568] kdmflush S ffff9c6a37fe1040 0 18060 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.092841] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.095468] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.100641] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.106484] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592403.112691] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.117762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.124034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.130668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.136936] bioset S ffff9c8a3514a080 0 18061 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.144210] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.146855] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.152000] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.157854] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592403.164037] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.169091] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.175380] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.181996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.188269] kdmwork-253:49 S ffff9c8a3514b0c0 0 18062 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.195556] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.198185] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.203333] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592403.209352] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592403.215015] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.220069] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.226357] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.232973] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.239239] kdmflush S ffff9c8a3514c100 0 18065 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.246527] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.249152] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.254292] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.260156] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592403.266341] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.271401] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.277681] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.284303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.290587] bioset S ffff9c8a22be8000 0 18066 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.297860] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.300484] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.305638] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.311475] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:38 2020][8592403.317655] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.322734] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.329000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.335615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.341903] kdmwork-253:50 S ffff9c8a22be9040 0 18067 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.349174] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.351803] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.356950] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:38 2020][8592403.362979] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:38 2020][8592403.368644] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:38 2020][8592403.373703] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.379982] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:38 2020][8592403.386600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:38 2020][8592403.392874] kworker/24:1H S ffff9c5a39d54100 0 18068 2 0x00000000 [Tue Mar 17 17:58:38 2020][8592403.400165] Call Trace: [Tue Mar 17 17:58:38 2020][8592403.402799] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:38 2020][8592403.408986] [] schedule+0x29/0x70 [Tue Mar 17 17:58:38 2020][8592403.414149] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:38 2020][8592403.419895] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:39 2020][8592403.426595] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.431677] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.437950] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.444562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.450842] kworker/16:1H S ffff9c5a39d56180 0 18071 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.458117] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.460744] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:39 2020][8592403.466931] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.472070] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.477816] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:39 2020][8592403.484530] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.489602] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.495870] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.502503] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.508774] kworker/12:1H S ffff9c5a39d55140 0 18080 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.516045] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.518671] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:39 2020][8592403.524866] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.530008] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.535754] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:39 2020][8592403.542469] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.547537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.553808] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.560443] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.566712] kworker/5:1H S ffff9c6a2f40b0c0 0 18103 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.573992] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.576618] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:39 2020][8592403.582818] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.587960] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.593707] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:39 2020][8592403.600421] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.605490] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.611761] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.618395] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.624666] kdmflush S ffff9c6a35a54100 0 18132 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.631946] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.634581] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.639757] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.645596] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592403.651776] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.656851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.663132] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.669750] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.676031] bioset S ffff9c6a35a52080 0 18134 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.683305] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.685939] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.691078] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.696918] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592403.703099] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.708151] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.714434] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.721065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.727332] kdmwork-253:51 S ffff9c8a35509040 0 18135 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.734627] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.737257] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.742402] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:39 2020][8592403.748408] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:39 2020][8592403.754068] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.759134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.765404] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.772030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.778316] kdmflush S ffff9c8a22bee180 0 18150 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.785592] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.788238] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.793382] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.799223] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592403.805411] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.810485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.816756] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.823376] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.829658] bioset S ffff9c8a22bed140 0 18151 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.836950] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.839582] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.844722] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.850569] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592403.856751] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.861805] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.868084] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.874700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.880967] kdmwork-253:52 S ffff9c8a22bec100 0 18152 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.888259] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.890890] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.896037] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:39 2020][8592403.902070] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:39 2020][8592403.907728] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.912782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.919063] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.925678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.931944] kdmflush S ffff9c8a22beb0c0 0 18168 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.939234] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.941866] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.947009] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592403.952863] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592403.959044] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592403.964096] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.970379] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592403.976994] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592403.983261] bioset S ffff9c8a22bea080 0 18169 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592403.990558] Call Trace: [Tue Mar 17 17:58:39 2020][8592403.993193] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592403.998339] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.004186] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.010370] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.015423] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.021711] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.028328] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.034616] kdmwork-253:53 S ffff9c8a1fef8000 0 18170 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.041887] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.044517] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.049667] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:39 2020][8592404.055678] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:39 2020][8592404.061351] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.066423] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.072692] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.079320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.085589] kdmflush S ffff9c8a1fef9040 0 18179 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.092860] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.095485] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.100642] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.106476] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.112677] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.117761] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.124034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.130661] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.136930] bioset S ffff9c8a1fefa080 0 18180 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.144210] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.146849] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.151991] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.157826] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.164013] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.169079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.175363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.181978] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.188259] kdmwork-253:54 S ffff9c8a1fefb0c0 0 18181 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.195529] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.198161] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.203308] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:39 2020][8592404.209315] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:39 2020][8592404.214983] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.220059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.226327] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.232939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.239232] kdmflush S ffff9c8a1fefc100 0 18189 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.246512] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.249137] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.254277] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.260110] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.266287] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.271356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.277637] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.284256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.290557] bioset S ffff9c8a1fefd140 0 18191 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.297836] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.300464] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.305610] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.311444] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.317631] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.322702] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.328976] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.335589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.341875] kdmwork-253:55 S ffff9c8a1fefe180 0 18192 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.349166] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.351794] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.356933] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:39 2020][8592404.362974] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:39 2020][8592404.368634] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:39 2020][8592404.373691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.379977] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:39 2020][8592404.386591] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:39 2020][8592404.392858] kdmflush S ffff9c8a3a00b0c0 0 18221 2 0x00000000 [Tue Mar 17 17:58:39 2020][8592404.400148] Call Trace: [Tue Mar 17 17:58:39 2020][8592404.402781] [] schedule+0x29/0x70 [Tue Mar 17 17:58:39 2020][8592404.407920] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:39 2020][8592404.413770] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:39 2020][8592404.419952] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.425012] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.431300] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.437916] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.444183] bioset S ffff9c8a3a009040 0 18222 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.451482] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.454114] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.459255] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.465110] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.471291] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.476345] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.482625] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.489240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.495506] kdmwork-253:56 S ffff9c8a3a00e180 0 18223 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.502792] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.505420] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.510561] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592404.516581] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592404.522244] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.527297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.533584] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.540202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.546477] kdmflush S ffff9c8a3a00a080 0 18231 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.553774] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.556397] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.561538] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.567386] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.573581] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.578640] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.584926] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.591543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.597818] bioset S ffff9c8a2e980000 0 18232 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.605116] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.607750] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.612898] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.618731] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.624918] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.629999] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.636274] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.642893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.649174] kdmwork-253:57 S ffff9c8a2e981040 0 18233 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.656444] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.659075] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.664222] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592404.670229] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592404.675897] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.680957] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.687251] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.693871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.700145] kdmflush S ffff9c8a2e982080 0 18241 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.707436] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.710069] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.715208] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.721040] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.727219] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.732288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.738556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.745169] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.751463] bioset S ffff9c8a2e9830c0 0 18242 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.758743] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.761375] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.766514] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.772347] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.778528] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.783603] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.789872] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.796485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.802779] kdmwork-253:58 S ffff9c8a2e984100 0 18243 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.810055] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.812685] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.817831] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592404.823837] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592404.829495] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.834563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.840832] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.847445] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.853723] kdmflush S ffff9c8a2e985140 0 18266 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.861018] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.863644] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.868792] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.874649] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.880829] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.885883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.892173] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.898788] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.905062] bioset S ffff9c8a2e986180 0 18267 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.912353] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.914985] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.920123] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592404.925986] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592404.932171] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.937222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.943504] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592404.950121] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.956393] kdmwork-253:59 S ffff9c8a33990000 0 18268 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592404.963679] Call Trace: [Tue Mar 17 17:58:40 2020][8592404.966307] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592404.971448] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592404.977469] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592404.983131] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592404.988184] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592404.994465] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.001079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.007345] kdmflush S ffff9c8a393be180 0 18275 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.014636] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.017269] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.022408] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.028257] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.034440] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.039499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.045780] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.052397] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.058671] bioset S ffff9c8a393bb0c0 0 18276 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.065960] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.068593] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.073732] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.079568] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.085766] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.090838] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.097108] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.103734] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.110004] kdmwork-253:60 S ffff9c8a393b9040 0 18277 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.117298] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.119941] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.125082] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592405.131089] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592405.136748] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.141829] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.148104] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.154723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.161003] kdmflush S ffff9c8a393ba080 0 18283 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.168279] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.170913] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.176051] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.181885] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.188065] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.193133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.199415] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.206030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.212312] bioset S ffff9c8a2e7bb0c0 0 18284 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.219586] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.222211] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.227352] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.233207] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.239389] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.244443] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.250731] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.257361] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.263630] kdmwork-253:61 S ffff9c8a2e7b8000 0 18285 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.270907] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.273536] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.278675] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:40 2020][8592405.284682] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:40 2020][8592405.290340] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.295408] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.301677] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.308290] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.314583] kdmflush S ffff9c7a2de1d140 0 18291 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.321864] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.324497] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.329646] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.335486] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.341665] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.346732] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.353003] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.359623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.365910] bioset S ffff9c7a2de1b0c0 0 18292 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.373205] Call Trace: [Tue Mar 17 17:58:40 2020][8592405.375839] [] schedule+0x29/0x70 [Tue Mar 17 17:58:40 2020][8592405.380987] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:40 2020][8592405.386843] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:40 2020][8592405.393024] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:40 2020][8592405.398076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.404355] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:40 2020][8592405.410973] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:40 2020][8592405.417238] kdmwork-253:62 S ffff9c7a2de1c100 0 18293 2 0x00000000 [Tue Mar 17 17:58:40 2020][8592405.424515] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.427146] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.432294] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592405.438336] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592405.444003] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.449062] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.455344] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.461958] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.468225] kdmflush S ffff9c7a2de1a080 0 18295 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.475515] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.478147] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.483289] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.489144] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.495325] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.500379] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.506660] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.513276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.519549] bioset S ffff9c7a2de18000 0 18296 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.526839] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.529474] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.534622] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.540464] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.546650] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.551719] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.557988] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.564609] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.570896] kdmwork-253:63 S ffff9c7a2de1e180 0 18297 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.578167] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.580798] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.585938] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592405.591951] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592405.597610] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.602695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.608964] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.615578] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.621867] kdmflush S ffff9c8a23ef2080 0 18299 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.629142] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.631776] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.636924] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.642769] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.648953] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.654014] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.660303] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.666919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.673185] bioset S ffff9c8a23ef6180 0 18300 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.680474] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.683100] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.688247] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.694094] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.700277] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.705331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.711633] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.718254] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.724527] kdmwork-253:64 S ffff9c8a23ef5140 0 18301 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.731822] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.734449] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.739588] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592405.745611] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592405.751273] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.756334] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.762621] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.769240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.775542] kdmflush S ffff9c8a23ef30c0 0 18318 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.782820] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.785452] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.790594] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.796450] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.802631] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.807684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.813967] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.820581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.826855] bioset S ffff9c8a23ef0000 0 18319 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.834143] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.836768] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.841910] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.847765] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.853946] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.859000] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.865296] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.871913] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.878178] kdmwork-253:65 S ffff9c8a17a58000 0 18320 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.885475] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.888103] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.893252] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592405.899280] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592405.904940] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.910009] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.916277] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.922891] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.929173] kdmflush S ffff9c8a17a59040 0 18322 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.936447] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.939080] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.944218] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592405.950065] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592405.956249] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592405.961303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.967591] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592405.974206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592405.980489] bioset S ffff9c8a17a5a080 0 18323 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592405.987762] Call Trace: [Tue Mar 17 17:58:41 2020][8592405.990396] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592405.995544] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.001391] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.007573] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.012626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.018908] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.025522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.031788] kdmwork-253:66 S ffff9c8a17a5b0c0 0 18324 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.039072] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.041702] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.046842] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592406.052877] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592406.058542] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.063595] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.069876] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.076490] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.082756] kdmflush S ffff9c8a17a5c100 0 18328 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.090046] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.092671] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.097813] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.103667] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.109867] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.114935] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.121218] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.127835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.134121] bioset S ffff9c8a17a5d140 0 18329 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.141397] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.144031] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.149178] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.155025] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.161210] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.166271] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.172573] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.179193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.185479] kdmwork-253:67 S ffff9c8a17a5e180 0 18330 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.192752] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.195395] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.200538] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592406.206560] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592406.212222] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.217282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.223575] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.230196] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.236470] kdmflush S ffff9c8a36588000 0 18334 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.243752] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.246385] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.251534] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.257388] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.263572] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.268634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.274921] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.281536] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.287828] bioset S ffff9c8a36589040 0 18335 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.295102] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.297735] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.302874] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.308721] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.314905] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.319967] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.326255] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.332870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.339134] kdmwork-253:68 S ffff9c8a3658a080 0 18336 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.346421] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.349050] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.354192] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:41 2020][8592406.360221] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:41 2020][8592406.365882] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.370937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.377222] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:41 2020][8592406.383839] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:41 2020][8592406.390121] kdmflush S ffff9c8a3658b0c0 0 18341 2 0x00000000 [Tue Mar 17 17:58:41 2020][8592406.397414] Call Trace: [Tue Mar 17 17:58:41 2020][8592406.400047] [] schedule+0x29/0x70 [Tue Mar 17 17:58:41 2020][8592406.405194] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:41 2020][8592406.411049] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:41 2020][8592406.417232] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:41 2020][8592406.422294] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.428583] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.435199] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.441474] bioset S ffff9c8a3658c100 0 18342 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.448772] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.451406] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.456553] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.462414] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.468600] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.473653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.479941] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.486558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.492832] kdmwork-253:69 S ffff9c8a3658d140 0 18343 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.500118] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.502749] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.507896] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592406.513922] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592406.519584] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.524652] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.530924] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.537543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.543826] kdmflush S ffff9c8a3658e180 0 18381 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.551100] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.553735] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.558881] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.564744] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.570929] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.575988] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.582268] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.588885] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.595151] bioset S ffff9c8a37300000 0 18382 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.602440] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.605068] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.610213] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.616060] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.622259] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.627313] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.633597] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.640219] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.646485] kdmwork-253:70 S ffff9c8a37301040 0 18384 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.653770] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.656397] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.661538] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592406.667558] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592406.673220] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.678274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.684568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.691187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.697453] kdmflush S ffff9c8a37302080 0 18388 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.704743] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.707377] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.712516] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.718362] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.724545] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.729598] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.735894] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.742512] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.748787] bioset S ffff9c8a373030c0 0 18389 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.756085] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.758719] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.763866] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.769714] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.775895] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.780963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.787233] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.793854] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.800157] kdmwork-253:71 S ffff9c8a37304100 0 18390 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.807431] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.810059] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.815201] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592406.821226] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592406.826889] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.831944] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.838224] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.844840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.851104] kdmflush S ffff9c8a37305140 0 18399 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.858397] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.861030] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.866178] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.872032] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.878215] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.883269] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.889556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.896174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.902446] bioset S ffff9c8a37306180 0 18400 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.909745] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.912379] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.917519] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592406.923374] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592406.929557] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.934617] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.940899] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.947514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.953780] kdmwork-253:72 S ffff9c8a20ad8000 0 18401 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592406.961065] Call Trace: [Tue Mar 17 17:58:42 2020][8592406.963702] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592406.968844] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592406.974871] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592406.980536] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592406.985613] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592406.991879] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592406.998492] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.004770] kdmflush S ffff9c8a20ad9040 0 18411 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.012049] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.014680] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.019820] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.025668] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.031849] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.036916] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.043187] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.049808] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.056089] bioset S ffff9c8a20ada080 0 18412 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.063362] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.065989] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.071138] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.077007] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.083192] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.088244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.094525] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.101141] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.107429] kdmwork-253:73 S ffff9c8a20adb0c0 0 18413 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.114717] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.117363] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.122505] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592407.128531] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592407.134223] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.139284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.145572] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.152188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.158456] kdmflush S ffffffffb861dfc0 0 18416 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.165751] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.168378] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.173526] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.179381] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.185563] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.190616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.196920] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.203537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.209805] bioset S ffff9c8a20add140 0 18417 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.217102] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.219728] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.224876] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.230732] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.236914] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.241976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.248275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.254889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.261162] kdmwork-253:74 S ffff9c8a20ade180 0 18418 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.268449] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.271078] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.276217] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:42 2020][8592407.282239] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:42 2020][8592407.287900] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.292961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.299242] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.305873] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.312141] kdmflush S ffff9c8a2ed50000 0 18427 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.319429] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.322056] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.327198] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.333052] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.339233] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.344288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.350575] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.357223] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.363491] bioset S ffff9c8a2ed51040 0 18429 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.370790] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.373424] [] schedule+0x29/0x70 [Tue Mar 17 17:58:42 2020][8592407.378571] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:42 2020][8592407.384418] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:42 2020][8592407.390601] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:42 2020][8592407.395653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.401933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:42 2020][8592407.408551] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:42 2020][8592407.414826] kdmwork-253:75 S ffff9c8a2ed52080 0 18430 2 0x00000000 [Tue Mar 17 17:58:42 2020][8592407.422128] Call Trace: [Tue Mar 17 17:58:42 2020][8592407.424757] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.429903] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592407.435925] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592407.441587] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.446653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.452922] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.459537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.465824] kdmflush S ffff9c8a2ed530c0 0 18439 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.473100] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.475726] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.480876] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.486730] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.492911] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.497964] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.504245] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.510860] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.517125] bioset S ffff9c8a2ed54100 0 18441 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.524417] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.527049] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.532188] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.538036] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.544217] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.549271] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.555553] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.562168] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.568456] kdmwork-253:76 S ffff9c8a2ed55140 0 18442 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.575729] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.578357] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.583495] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592407.589530] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592407.595197] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.600258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.606547] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.613164] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.619436] kdmflush S ffff9c7a34458000 0 18485 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.626727] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.629360] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.634500] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.640357] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.646552] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.651607] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.657890] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.664504] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.670768] bioset S ffff9c7a3445b0c0 0 18488 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.678062] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.680701] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.685842] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.691702] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.697888] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.702942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.709243] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.715864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.722136] kdmwork-253:77 S ffff9c7a3445e180 0 18489 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.729424] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.732053] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.737200] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592407.743220] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592407.748882] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.753936] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.760232] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.766849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.773116] kdmflush S ffff9c8a305f0000 0 18491 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.780413] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.783039] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.788180] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.794036] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.800218] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.805280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.811568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.818196] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.824466] bioset S ffff9c8a305f1040 0 18493 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.831754] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.834379] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.839520] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.845368] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.851550] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.856611] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.862902] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.869516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.875790] kdmwork-253:78 S ffff9c8a305f2080 0 18494 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.883092] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.885723] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.890870] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592407.896890] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592407.902552] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.907606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.913894] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.920511] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.926790] kdmflush S ffff9c8a305f4100 0 18504 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.934083] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.936708] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.941849] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.947711] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592407.953896] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592407.958969] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.965241] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592407.971859] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592407.978140] bioset S ffff9c8a339930c0 0 18505 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592407.985417] Call Trace: [Tue Mar 17 17:58:43 2020][8592407.988050] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592407.993190] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592407.999045] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.005228] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.010280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.016561] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.023176] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.029443] kdmwork-253:79 S ffff9c8a33992080 0 18507 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.036736] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.039365] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.044505] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592408.050525] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592408.056188] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.061255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.067526] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.074147] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.080436] kdmflush S ffff9c8a33995140 0 18515 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.087710] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.090358] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.095499] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592408.101362] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.107546] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.112600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.118903] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.125522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.131789] bioset S ffff9c8a33996180 0 18516 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.139086] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.141721] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.146868] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592408.152720] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.158920] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.163977] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.170266] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.176881] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.183156] kdmwork-253:80 S ffff9c8a2c2dc100 0 18517 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.190450] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.193081] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.198228] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592408.204256] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592408.209919] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.214978] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.221273] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.227893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.234159] kdmflush S ffff9c8a2c2db0c0 0 18523 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.241440] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.244067] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.249215] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592408.255069] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.261252] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.266303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.272597] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.279210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.285485] bioset S ffff9c8a2c2da080 0 18524 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.292783] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.295415] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.300553] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592408.306401] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.312586] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.317645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.323926] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.330558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.336826] kdmwork-253:81 S ffff9c8a2c2d9040 0 18525 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.344121] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.346747] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.351889] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:43 2020][8592408.357936] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:43 2020][8592408.363595] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.368651] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.374938] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:43 2020][8592408.381555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.387829] kdmflush S ffff9c8a2c2d8000 0 18529 2 0x00000000 [Tue Mar 17 17:58:43 2020][8592408.395127] Call Trace: [Tue Mar 17 17:58:43 2020][8592408.397763] [] schedule+0x29/0x70 [Tue Mar 17 17:58:43 2020][8592408.402910] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:43 2020][8592408.408765] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:43 2020][8592408.414948] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:43 2020][8592408.420008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:43 2020][8592408.426289] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.432906] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.439179] bioset S ffff9c8a2ed56180 0 18532 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.446479] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.449112] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.454259] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.460107] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.466293] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.471351] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.477641] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.484255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.490522] kdmwork-253:82 S ffff9c8a31a18000 0 18533 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.497817] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.500446] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.505592] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592408.511612] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592408.517274] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.522330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.528617] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.535232] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.541500] kdmflush S ffff9c8a31a19040 0 18541 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.548796] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.551423] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.556569] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.562418] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.568598] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.573653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.579940] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.586557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.592824] bioset S ffff9c8a305f5140 0 18542 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.600122] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.602756] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.607904] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.613771] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.619949] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.625001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.631283] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.637898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.644166] kdmwork-253:83 S ffff9c8a305f6180 0 18543 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.651477] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.654105] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.659244] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592408.665265] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592408.670942] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.675997] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.682278] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.688894] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.695173] kdmflush S ffff9c8a1fb50000 0 18551 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.702451] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.705084] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.710232] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.716087] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.722269] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.727324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.733623] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.740245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.746531] bioset S ffff9c8a1fb51040 0 18553 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.753809] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.756441] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.761582] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.767439] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.773619] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.778673] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.784967] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.791587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.797859] kdmwork-253:84 S ffff9c8a1fb52080 0 18555 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.805147] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.807774] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.812916] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592408.818943] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592408.824605] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.829674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.835941] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.842555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.848847] kdmflush S ffff9c8a1fb530c0 0 18560 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.856122] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.858755] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.863903] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.869756] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.875939] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.881001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.887279] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.893896] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.900162] bioset S ffff9c8a1fb54100 0 18561 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.907452] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.910078] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.915225] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592408.921073] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592408.927255] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.932306] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.938586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.945205] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.951478] kdmwork-253:85 S ffff9c8a1fb55140 0 18562 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592408.958772] Call Trace: [Tue Mar 17 17:58:44 2020][8592408.961402] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592408.966541] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592408.972569] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592408.978231] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592408.983285] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592408.989564] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592408.996180] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.002447] kdmflush S ffff9c8a1fb56180 0 18570 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.009737] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.012372] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.017519] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.023374] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.029557] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.034618] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.040897] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.047514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.053781] bioset S ffff9c8a31cb0000 0 18573 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.061070] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.063710] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.068853] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.074713] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.080899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.085962] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.092249] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.098864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.105146] kdmwork-253:86 S ffff9c8a31cb1040 0 18574 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.112418] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.115060] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.120217] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592409.126253] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592409.131918] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.136986] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.143254] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.149870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.156159] kdmflush S ffff9c8a31cb2080 0 18582 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.163434] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.166067] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.171216] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.177069] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.183252] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.188332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.194607] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.201228] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.207506] bioset S ffff9c8a31cb30c0 0 18585 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.214782] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.217409] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.222556] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.228402] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.234587] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.239648] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.245935] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.252553] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.258841] kdmwork-253:87 S ffff9c8a31cb4100 0 18586 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.266112] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.268754] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.273900] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:44 2020][8592409.279928] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:44 2020][8592409.285591] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.290665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.296949] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.303566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.309854] kdmflush S ffff9c8a31cb5140 0 18593 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.317129] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.319761] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.324902] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.330756] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.336940] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.342001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.348281] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.354913] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.361199] bioset S ffff9c8a31cb6180 0 18595 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.368497] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.371129] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.376268] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:44 2020][8592409.382118] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:44 2020][8592409.388298] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:44 2020][8592409.393350] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.399632] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:44 2020][8592409.406248] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:44 2020][8592409.412521] kdmwork-253:88 S ffff9c7a1ef32080 0 18596 2 0x00000000 [Tue Mar 17 17:58:44 2020][8592409.419817] Call Trace: [Tue Mar 17 17:58:44 2020][8592409.422446] [] schedule+0x29/0x70 [Tue Mar 17 17:58:44 2020][8592409.427594] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592409.433613] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592409.439276] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.444343] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.450612] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.457225] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.463512] kdmflush S ffff9c7a1ef35140 0 18661 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.470788] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.473440] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.478580] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.484434] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.490618] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.495671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.501950] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.508566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.514835] bioset S ffff9c8a31a1b0c0 0 18663 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.522131] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.524764] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.529904] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.535764] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.541952] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.547013] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.553303] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.559917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.566182] kdmwork-253:89 S ffff9c8a31a1c100 0 18664 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.573469] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.576099] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.581245] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592409.587268] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592409.592928] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.597982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.604270] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.610886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.617151] kdmflush S ffff9c8a31a1d140 0 18678 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.624442] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.627075] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.632216] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.638085] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.644272] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.649332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.655610] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.662228] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.668506] bioset S ffff9c8a31a1e180 0 18680 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.675785] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.678417] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.683557] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.689412] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.695610] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.700663] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.706946] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.713561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.719829] kdmwork-253:90 S ffff9c8a31a1a080 0 18681 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.727121] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.729750] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.734888] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592409.740912] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592409.746573] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.751635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.757935] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.764557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.770843] kdmflush S ffff9c8a3a31c100 0 18691 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.778119] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.780746] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.785895] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.791750] [] ? _raw_spin_unlock_irqrestore+0x15/0x20 [Tue Mar 17 17:58:45 2020][8592409.798712] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.804899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.809978] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.816246] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.822866] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.829154] bioset S ffff9c8a1eac5140 0 18692 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.836429] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.839057] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.844203] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.850051] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.856232] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.861286] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.867588] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.874206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.880474] kdmwork-253:91 S ffff9c8a1eac30c0 0 18694 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.887758] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.890390] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.895537] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592409.901563] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592409.907228] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.912288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.918570] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.925200] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.931468] kdmflush S ffff9c8a1eac2080 0 18698 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.938757] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.941384] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.946532] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592409.952387] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592409.958569] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592409.963620] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.969903] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592409.976518] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592409.982792] bioset S ffff9c8a1eac4100 0 18700 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592409.990100] Call Trace: [Tue Mar 17 17:58:45 2020][8592409.992733] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592409.997874] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.003728] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.009911] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.014965] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.021252] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.027869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.034157] kdmwork-253:92 S ffff9c8a1eac0000 0 18701 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.041446] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.044074] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.049215] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592410.055243] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592410.060905] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.065957] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.072240] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.078854] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.085120] kdmflush S ffff9c8a305f30c0 0 18812 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.092410] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.095043] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.100183] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.106044] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.112230] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.117283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.123577] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.130198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.136487] bioset S ffff9c8a33991040 0 18814 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.143777] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.146403] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.151558] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.157406] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.163590] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.168667] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.174937] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.181557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.187846] kdmwork-253:93 S ffff9c8a2e8b1040 0 18815 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.195117] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.197745] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.202885] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592410.208925] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592410.214585] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.219646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.225926] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.232543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.238816] kdmflush S ffff9c8a31a74100 0 19042 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.246107] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.248741] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.253887] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.259735] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.265931] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.270989] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.277275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.283892] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.290159] bioset S ffff9c7a36a2d140 0 19045 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.297457] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.300090] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.305229] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.311078] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.317260] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.322322] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.328622] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.335244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.341532] kdmwork-253:94 S ffff9c7a3714c100 0 19046 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.348802] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.351447] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.356588] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:45 2020][8592410.362611] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:45 2020][8592410.368273] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.373332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.379625] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:45 2020][8592410.386244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.392525] kdmflush S ffff9c8a36359040 0 19065 2 0x00000000 [Tue Mar 17 17:58:45 2020][8592410.399803] Call Trace: [Tue Mar 17 17:58:45 2020][8592410.402435] [] schedule+0x29/0x70 [Tue Mar 17 17:58:45 2020][8592410.407576] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:45 2020][8592410.413431] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:45 2020][8592410.419613] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:45 2020][8592410.424665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:45 2020][8592410.430949] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.437576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.443846] bioset S ffff9c8a3635a080 0 19070 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.451143] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.453770] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.458918] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.464774] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.470954] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.476008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.482288] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.488903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.495196] kdmwork-253:95 S ffff9c8a3635b0c0 0 19071 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.502463] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.505093] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.510232] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592410.516253] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592410.521915] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.526968] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.533247] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.539863] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.546131] kdmflush S ffff9c8a3635e180 0 19144 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.553445] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.556072] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.561220] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.567074] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.573255] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.578324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.584594] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.591215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.597502] bioset S ffff9c8a2e8b5140 0 19145 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.604779] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.607411] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.612553] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.618422] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.624608] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.629668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.635948] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.642563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.648830] kdmwork-253:96 S ffff9c8a33652080 0 19146 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.656117] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.658746] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.663892] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592410.669914] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592410.675576] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.680627] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.686910] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.693524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.699789] kdmflush S ffff9c8a336530c0 0 19157 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.707080] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.709708] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.714855] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.720725] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.726909] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.731970] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.738249] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.744868] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.751141] bioset S ffff9c8a33650000 0 19158 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.758432] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.761064] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.766204] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.772052] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.778246] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.783305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.789593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.796210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.802498] kdmwork-253:97 S ffff9c8a2c2de180 0 19159 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.809769] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.812397] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.817536] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592410.823558] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592410.829219] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.834272] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.840566] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.847186] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.853450] kdmflush S ffff9c8a2f265140 0 19254 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.860742] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.863377] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.868524] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.874371] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.880553] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.885605] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.891898] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.898519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.904786] bioset S ffff9c8a2f262080 0 19258 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.912083] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.914717] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.919855] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592410.925702] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592410.931894] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.936962] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.943231] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592410.949869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.956137] kdmwork-253:98 S ffff9c8a2f260000 0 19259 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592410.963411] Call Trace: [Tue Mar 17 17:58:46 2020][8592410.966054] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592410.971199] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592410.977226] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592410.982889] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592410.987942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592410.994221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.000840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.007112] kdmflush S ffff9c8a2f264100 0 19368 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.014412] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.017046] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.022195] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.028050] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.034231] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.039284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.045566] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.052182] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.058456] bioset S ffff9c8a3764ea40 0 19373 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.065761] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.068396] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.073543] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.079390] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.085573] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.090626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.096914] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.103530] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.109797] kdmwork-253:99 S ffff9c7a30aac100 0 19375 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.117090] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.119734] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.124876] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592411.130910] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592411.136577] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.141645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.147928] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.154543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.160830] kdmflush S ffff9c7a30aa9040 0 19389 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.168108] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.170740] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.175880] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.181729] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.187908] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.192962] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.199243] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.205860] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.212134] bioset S ffff9c7a30aae180 0 19390 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.219423] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.222057] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.227195] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.233067] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.239250] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.244305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.250592] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.257210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.263483] kdmwork-253:100 S ffff9c8a2c2dd140 0 19392 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.270770] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.273398] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.278540] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:46 2020][8592411.284566] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:46 2020][8592411.290243] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.295298] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.301580] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.308195] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.314463] kdmflush S ffff9c8a1f272080 0 19612 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.321760] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.324391] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.329532] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.335381] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.341562] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.346616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.352918] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.359539] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.365827] bioset S ffff9c8a2538c100 0 19613 2 0x00000000 [Tue Mar 17 17:58:46 2020][8592411.373102] Call Trace: [Tue Mar 17 17:58:46 2020][8592411.375748] [] schedule+0x29/0x70 [Tue Mar 17 17:58:46 2020][8592411.380892] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:46 2020][8592411.386738] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:46 2020][8592411.392921] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:46 2020][8592411.397973] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:46 2020][8592411.404267] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:46 2020][8592411.410888] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.417163] kdmwork-253:101 S ffff9c8a2538b0c0 0 19614 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.424456] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.427084] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.432224] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592411.438243] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592411.443907] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.448961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.455250] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.461865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.468167] kdmflush S ffff9c8a32080000 0 19662 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.475448] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.478096] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.483237] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.489099] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.495285] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.500362] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.506630] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.513250] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.519544] bioset S ffff9c8a32081040 0 19663 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.526822] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.529457] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.534605] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.540461] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.546642] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.551697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.557986] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.564599] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.570866] kdmwork-253:102 S ffff9c8a32082080 0 19664 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.578161] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.580791] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.585937] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592411.591959] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592411.597621] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.602682] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.608964] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.615594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.621863] kvm-irqfd-clean S ffff9c8a2fd0c100 0 19703 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.629159] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.631785] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.636931] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.642778] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.648963] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.654023] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.660305] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.666921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.673195] kdmflush S ffff9c8a3550d140 0 19991 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.680485] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.683119] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.688265] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.694113] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.700295] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.705349] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.711637] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.718255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.724528] bioset S ffff9c8a2e7be180 0 19993 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.731818] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.734450] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.739591] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.745460] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.751644] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.756718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.762991] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.769611] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.775892] kdmwork-253:103 S ffff9c8a2e5ce180 0 19994 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.783164] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.785791] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.790931] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592411.796953] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592411.802629] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.807684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.813964] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.820580] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.826847] kdmflush S ffff9c8a2e5c8000 0 20036 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.834135] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.836761] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.841901] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.847748] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.853930] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.858982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.865277] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.871897] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.878172] bioset S ffff9c8a33651040 0 20038 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.885469] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.888103] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.893251] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592411.899099] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592411.905281] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.910342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.916636] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.923256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.929529] kdmwork-253:104 S ffff9c8a33654100 0 20039 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.936815] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.939443] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.944584] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592411.950605] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592411.956268] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592411.961343] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.967613] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592411.974233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592411.980527] kdmflush S ffff9c8a320830c0 0 20094 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592411.987807] Call Trace: [Tue Mar 17 17:58:47 2020][8592411.990438] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592411.995580] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.001437] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.007618] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.012679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.018969] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.025583] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.031850] bioset S ffff9c8a2e7bc100 0 20095 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.039148] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.041782] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.046922] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.052778] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.058958] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.064013] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.070301] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.076917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.083191] kdmwork-253:105 S ffff9c8a3550c100 0 20096 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.090485] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.093116] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.098263] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592412.104290] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592412.109955] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.115015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.121295] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.127925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.134211] kdmflush S ffff9c8a3635c100 0 20661 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.141483] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.144124] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.149266] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.155115] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.161295] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.166350] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.172638] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.179253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.185527] bioset S ffff9c8a36358000 0 20669 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.192818] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.195452] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.200601] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.206456] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.212638] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.217700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.223986] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.230603] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.236869] kdmwork-253:106 S ffff9c8a3635d140 0 20670 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.244155] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.246785] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.251932] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:47 2020][8592412.257967] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:47 2020][8592412.263633] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.268698] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 17:58:47 2020][8592412.274982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.281253] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.287873] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.294154] kdmflush S ffff9c7a3693b0c0 0 21023 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.301428] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.304055] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.309202] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.315063] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.321248] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.326300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.332584] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.339197] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.345463] bioset S ffff9c7a36938000 0 21028 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.352743] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.355370] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.360518] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:47 2020][8592412.366366] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:47 2020][8592412.372561] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:47 2020][8592412.377619] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.383905] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:47 2020][8592412.390523] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:47 2020][8592412.396797] kdmwork-253:107 S ffff9c7a2f4e0000 0 21029 2 0x00000000 [Tue Mar 17 17:58:47 2020][8592412.404083] Call Trace: [Tue Mar 17 17:58:47 2020][8592412.406714] [] schedule+0x29/0x70 [Tue Mar 17 17:58:47 2020][8592412.411859] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592412.417878] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592412.423541] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.428594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.434889] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.441510] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.447782] kdmflush S ffff9c8a1f7da080 0 24526 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.455073] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.457707] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.462853] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.468702] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.474883] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.479951] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.486235] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.492849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.499130] bioset S ffff9c6a24f730c0 0 24527 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.506406] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.509038] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.514179] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.520036] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.526218] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.531277] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.537559] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.544175] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.550476] kdmwork-253:108 S ffff9c6a24f71040 0 24528 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.557755] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.560394] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.565537] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592412.571559] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592412.577219] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.582273] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.588555] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.595170] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.601445] kdmflush S ffff9c8a36f4a080 0 25205 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.608742] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.611376] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.616515] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.622363] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.628547] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.633605] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.639887] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.646503] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.652779] bioset S ffff9c8a36f4e180 0 25206 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.660094] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.662726] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.667874] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.673725] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.679905] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.684967] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.691255] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.697871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.704145] kdmwork-253:109 S ffff9c8a36f4c100 0 25207 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.711440] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.714069] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.719218] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592412.725258] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592412.730924] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.735991] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.742260] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.748874] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.755163] kworker/13:1H S ffff9c6a30a25140 0 25210 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.762437] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.765065] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:48 2020][8592412.771251] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.776406] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.782155] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:48 2020][8592412.788875] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.793932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.800206] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.806836] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.813103] kdmflush S ffff9c5a39222bc0 0 25232 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.820375] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.823015] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.828158] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.834012] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.840194] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.845247] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.851529] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.858144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.864412] bioset S ffff9c6a30a22080 0 25233 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.871714] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.874340] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.879481] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.885327] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.891510] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.896564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.902852] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.909469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.915742] kdmwork-253:110 S ffff9c6a2f216180 0 25234 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.923029] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.925658] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.930798] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592412.936828] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592412.942502] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.947559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.953849] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592412.960463] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592412.966741] kdmflush S ffff9c6a2f2130c0 0 25245 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592412.974020] Call Trace: [Tue Mar 17 17:58:48 2020][8592412.976654] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592412.981799] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592412.987650] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592412.993829] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592412.998882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.005177] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.011794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.018062] bioset S ffff9c6a2f210000 0 25246 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.025351] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.027978] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.033126] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.038982] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592413.045162] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.050217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.056517] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.063129] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.069409] kdmwork-253:111 S ffff9c6a33f85140 0 25247 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.076682] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.079324] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.084466] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592413.090487] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592413.096151] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.101212] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.107500] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.114117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.120418] kdmflush S ffff9c6a39cde180 0 25377 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.127718] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.130348] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.135489] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.141358] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592413.147542] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.152596] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.158876] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.165494] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.171767] bioset S ffff9c6a39cd8000 0 25378 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.179057] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.181692] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.186839] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.192695] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592413.198875] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.203927] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.210210] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.216825] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.223091] kdmwork-253:112 S ffff9c8a2bfe4100 0 25379 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.230385] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.233014] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.238152] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:48 2020][8592413.244175] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:48 2020][8592413.249838] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.254899] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.261186] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.267802] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.274067] kworker/40:1H S ffff9c4b6976d140 0 25524 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.281375] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.284002] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 17:58:48 2020][8592413.290189] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.295356] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.301110] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 17:58:48 2020][8592413.307816] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.312883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.319153] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.325789] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.332057] kdmflush S ffff9c8a1f7dc100 0 26032 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.339330] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.341976] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.347122] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.352978] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592413.359179] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.364240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.370527] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:48 2020][8592413.377142] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:48 2020][8592413.383410] bioset S ffff9c8a1f7de180 0 26034 2 0x00000000 [Tue Mar 17 17:58:48 2020][8592413.390707] Call Trace: [Tue Mar 17 17:58:48 2020][8592413.393341] [] schedule+0x29/0x70 [Tue Mar 17 17:58:48 2020][8592413.398489] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:48 2020][8592413.404345] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:48 2020][8592413.410527] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:48 2020][8592413.415589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.421876] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.428493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.434767] kdmwork-253:113 S ffff9c6a24f72080 0 26036 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.442052] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.444683] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.449831] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592413.455872] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592413.461538] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.466606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.472876] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.479497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.485783] kdmflush S ffff9c8a327d5140 0 26321 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.493060] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.495693] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.500834] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.506689] [] ? _raw_spin_unlock_irqrestore+0x15/0x20 [Tue Mar 17 17:58:49 2020][8592413.513665] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.519850] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.524924] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.531193] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.537805] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.544087] bioset S ffff9c6a35a56180 0 26324 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.551362] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.553996] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.559143] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.564993] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.571187] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.576243] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.582522] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.589138] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.595404] kdmwork-253:114 S ffff9c6a35a50000 0 26325 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.602690] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.605320] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.610468] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592413.616489] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592413.622151] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.627213] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.633516] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.640135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.646407] kdmflush S ffff9c6a22ab4100 0 27524 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.653699] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.656333] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.661480] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.667337] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.673519] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.678580] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.684882] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.691502] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.697776] bioset S ffff9c6a22ab0000 0 27525 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.705065] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.707690] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.712833] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.718687] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.724869] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.729930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.736211] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.742840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.749109] kdmwork-253:115 S ffff9c7a36e3c100 0 27526 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.756394] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.759025] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.764171] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592413.770193] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592413.775856] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.780916] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.787198] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.793811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.800078] kdmflush S ffff9c6a23659040 0 27772 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.807376] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.810003] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.815150] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.820996] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.827178] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.832233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.838519] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.845137] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.851425] bioset S ffff9c6a2365c100 0 27773 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.858723] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.861354] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.866501] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.872355] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.878539] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.883613] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.889883] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.896510] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.902778] kdmwork-253:116 S ffff9c6a2365a080 0 27774 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.910049] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.912677] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.917830] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592413.923855] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592413.929517] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.934593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.940861] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.947482] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.953763] kdmflush S ffff9c6a2365d140 0 27829 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592413.961038] Call Trace: [Tue Mar 17 17:58:49 2020][8592413.963669] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592413.968811] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592413.974659] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592413.980840] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592413.985893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592413.992175] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592413.998790] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.005065] bioset S ffff9c6a2365b0c0 0 27831 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.012361] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.014987] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.020126] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592414.025989] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592414.032173] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.037226] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.043515] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.050131] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.056407] kdmwork-253:117 S ffff9c7a20e82080 0 27832 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.063700] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.066328] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.071467] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592414.077488] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592414.083166] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.088221] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.094502] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.101117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.107381] kdmflush S ffff9c6a28ee6180 0 30889 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.114672] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.117306] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.122447] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592414.128301] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592414.134484] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.139537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.145838] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.152473] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.158741] bioset S ffff9c6a28ee2080 0 30890 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.166032] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.168665] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.173804] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592414.179652] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592414.185836] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.190897] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.197198] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.203819] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.210106] kdmwork-253:118 S ffff9c6a28ee30c0 0 30891 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.217379] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.220008] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.225156] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592414.231184] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592414.236847] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.241921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.248190] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.254804] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.261097] kdmflush S ffff9c5a277d5140 0 31129 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.268376] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.271002] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.276150] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592414.281999] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592414.288179] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.293232] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.299513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.306129] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.312394] bioset S ffff9c5a277d4100 0 31153 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.319684] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.322318] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.327466] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:49 2020][8592414.333314] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:49 2020][8592414.339494] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.344561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.350831] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.357466] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.363749] kdmwork-253:119 S ffff9c8a36f4b0c0 0 31154 2 0x00000000 [Tue Mar 17 17:58:49 2020][8592414.371023] Call Trace: [Tue Mar 17 17:58:49 2020][8592414.373652] [] schedule+0x29/0x70 [Tue Mar 17 17:58:49 2020][8592414.378815] [] kthread_worker_fn+0xf7/0x180 [Tue Mar 17 17:58:49 2020][8592414.384838] [] ? kthread_stop+0xf0/0xf0 [Tue Mar 17 17:58:49 2020][8592414.390500] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:49 2020][8592414.395575] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.401844] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:49 2020][8592414.408465] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:49 2020][8592414.414745] auditd S ffff9c6a02686180 0 50139 1 0x00000000 [Tue Mar 17 17:58:50 2020][8592414.422017] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.424650] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:50 2020][8592414.431354] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.436516] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:50 2020][8592414.443654] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:50 2020][8592414.449579] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:50 2020][8592414.456900] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592414.463432] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:50 2020][8592414.468667] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:50 2020][8592414.474433] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:50 2020][8592414.480184] [] ? SyS_write+0xa4/0xf0 [Tue Mar 17 17:58:50 2020][8592414.485592] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592414.491779] auditd S ffff9c7a2b525140 0 50140 1 0x00000000 [Tue Mar 17 17:58:50 2020][8592414.499048] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.501675] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.506833] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:50 2020][8592414.513022] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:50 2020][8592414.518516] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:58:50 2020][8592414.524198] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:50 2020][8592414.529540] [] ? jbd2_complete_transaction+0x52/0xa0 [jbd2] [Tue Mar 17 17:58:50 2020][8592414.536932] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:50 2020][8592414.542259] [] ? do_fsync+0x78/0xb0 [Tue Mar 17 17:58:50 2020][8592414.547574] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592414.553754] rpciod S ffff9c7a322f1040 0 50143 2 0x00000080 [Tue Mar 17 17:58:50 2020][8592414.561044] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.563676] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.568818] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:50 2020][8592414.574674] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:50 2020][8592414.580854] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:50 2020][8592414.585908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:50 2020][8592414.592190] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:50 2020][8592414.598819] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:50 2020][8592414.605088] xprtiod S ffff9c7a322f0000 0 50144 2 0x00000080 [Tue Mar 17 17:58:50 2020][8592414.612384] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.615013] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.620159] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:50 2020][8592414.626015] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:50 2020][8592414.632196] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:50 2020][8592414.637251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:50 2020][8592414.643539] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:50 2020][8592414.650156] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:50 2020][8592414.656431] sssd S ffff9c6a22ab6180 0 50166 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592414.663724] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.666354] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:50 2020][8592414.673062] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.678224] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:50 2020][8592414.685359] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:50 2020][8592414.691287] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:50 2020][8592414.698615] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592414.705147] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:50 2020][8592414.711419] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:50 2020][8592414.717008] [] ? task_scan_min+0x3e/0x60 [Tue Mar 17 17:58:50 2020][8592414.722755] [] ? should_numa_migrate_memory+0x5b/0x150 [Tue Mar 17 17:58:50 2020][8592414.729716] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.735224] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.740712] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.746201] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.751708] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.757196] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.762697] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.768186] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.773675] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:50 2020][8592414.779527] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:50 2020][8592414.784757] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592414.790949] polkitd S ffff9c6a22ab2080 0 50170 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592414.798223] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.800850] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.805990] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:50 2020][8592414.813225] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:50 2020][8592414.819061] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592414.825595] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:50 2020][8592414.831884] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:50 2020][8592414.837461] [] ? __schedule+0x32c/0x860 [Tue Mar 17 17:58:50 2020][8592414.843128] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:50 2020][8592414.849418] [] ? __switch_to+0x151/0x580 [Tue Mar 17 17:58:50 2020][8592414.855166] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:58:50 2020][8592414.860826] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:50 2020][8592414.866767] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.872256] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592414.877746] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:50 2020][8592414.884203] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:50 2020][8592414.890215] [] ? inotify_free_event+0xe/0x10 [Tue Mar 17 17:58:50 2020][8592414.896332] [] ? kfree+0x106/0x140 [Tue Mar 17 17:58:50 2020][8592414.901562] [] ? list_del+0xd/0x30 [Tue Mar 17 17:58:50 2020][8592414.906794] [] ? inotify_read+0x2ef/0x420 [Tue Mar 17 17:58:50 2020][8592414.912648] [] ? eventfd_ctx_read+0x67/0x1e0 [Tue Mar 17 17:58:50 2020][8592414.918744] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:50 2020][8592414.923975] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 17:58:50 2020][8592414.930519] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592414.936702] gmain S ffff9c8a327d30c0 0 50202 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592414.943970] Call Trace: [Tue Mar 17 17:58:50 2020][8592414.946602] [] ? get_page_from_freelist+0x412/0xa70 [Tue Mar 17 17:58:50 2020][8592414.953323] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592414.958480] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:50 2020][8592414.965703] [] ? zone_statistics+0x88/0xa0 [Tue Mar 17 17:58:50 2020][8592414.971641] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:50 2020][8592414.977478] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592414.984004] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:50 2020][8592414.990286] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:50 2020][8592414.995864] [] ? radix_tree_lookup_slot+0x22/0x50 [Tue Mar 17 17:58:50 2020][8592415.002397] [] ? __find_get_page+0x1e/0xa0 [Tue Mar 17 17:58:50 2020][8592415.008340] [] ? filemap_fault+0x17e/0x490 [Tue Mar 17 17:58:50 2020][8592415.014261] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592415.019748] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592415.025256] [] ? do_read_fault.isra.61+0x139/0x1b0 [Tue Mar 17 17:58:50 2020][8592415.031873] [] ? handle_pte_fault+0x2f4/0xd10 [Tue Mar 17 17:58:50 2020][8592415.038061] [] ? eventfd_ctx_read+0x67/0x1e0 [Tue Mar 17 17:58:50 2020][8592415.044176] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:50 2020][8592415.049405] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 17:58:50 2020][8592415.055939] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592415.062133] gdbus S ffff9c8a163c9040 0 50223 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592415.069422] Call Trace: [Tue Mar 17 17:58:50 2020][8592415.072054] [] ? __kmalloc_reserve.isra.32+0x31/0x90 [Tue Mar 17 17:58:50 2020][8592415.078846] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592415.084008] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:50 2020][8592415.091231] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:50 2020][8592415.097072] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:50 2020][8592415.102926] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592415.109455] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:50 2020][8592415.115725] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:50 2020][8592415.121320] [] ? sock_sendmsg+0xb6/0xf0 [Tue Mar 17 17:58:50 2020][8592415.126981] [] ? sock_recvmsg+0xc5/0x100 [Tue Mar 17 17:58:50 2020][8592415.132763] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:58:50 2020][8592415.138431] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592415.143923] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592415.149437] [] ? ___sys_sendmsg+0x3e9/0x400 [Tue Mar 17 17:58:50 2020][8592415.155451] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:50 2020][8592415.161899] [] ? default_wake_function+0x12/0x20 [Tue Mar 17 17:58:50 2020][8592415.168362] [] ? pollwake+0x73/0x90 [Tue Mar 17 17:58:50 2020][8592415.173677] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:50 2020][8592415.179431] [] ? eventfd_ctx_read+0x67/0x1e0 [Tue Mar 17 17:58:50 2020][8592415.185539] [] ? __sys_sendmsg+0x85/0x90 [Tue Mar 17 17:58:50 2020][8592415.191288] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:50 2020][8592415.196524] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 17:58:50 2020][8592415.203071] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592415.209253] polkitd S ffffffffb856a9bc 0 50225 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592415.216540] Call Trace: [Tue Mar 17 17:58:50 2020][8592415.219182] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592415.224326] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:50 2020][8592415.231567] [] ? zone_statistics+0x88/0xa0 [Tue Mar 17 17:58:50 2020][8592415.237489] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:50 2020][8592415.243322] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:50 2020][8592415.249850] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:50 2020][8592415.256138] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:50 2020][8592415.261715] [] ? __alloc_pages_nodemask+0x176/0x420 [Tue Mar 17 17:58:50 2020][8592415.268421] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:50 2020][8592415.273921] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:50 2020][8592415.280364] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:50 2020][8592415.286383] [] ? wake_up_q+0x5b/0x80 [Tue Mar 17 17:58:50 2020][8592415.291787] [] ? futex_wake+0x90/0x180 [Tue Mar 17 17:58:50 2020][8592415.297367] [] ? eventfd_ctx_read+0x67/0x1e0 [Tue Mar 17 17:58:50 2020][8592415.303474] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:50 2020][8592415.308703] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 17:58:50 2020][8592415.315231] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592415.321434] JS GC Helper S ffff9c8a163cb0c0 0 50227 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592415.328704] Call Trace: [Tue Mar 17 17:58:50 2020][8592415.331333] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592415.336474] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:50 2020][8592415.342676] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:50 2020][8592415.348164] [] ? handle_pte_fault+0x2f4/0xd10 [Tue Mar 17 17:58:50 2020][8592415.354358] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:50 2020][8592415.359674] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:50 2020][8592415.365010] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:50 2020][8592415.371191] JS Sour~ Thread S ffff9c8a163ce180 0 50229 1 0x00000080 [Tue Mar 17 17:58:50 2020][8592415.378460] Call Trace: [Tue Mar 17 17:58:50 2020][8592415.381101] [] schedule+0x29/0x70 [Tue Mar 17 17:58:50 2020][8592415.386246] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:50 2020][8592415.392449] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:50 2020][8592415.397940] [] ? lru_cache_add+0xe/0x10 [Tue Mar 17 17:58:50 2020][8592415.403605] [] ? page_add_new_anon_rmap+0xb8/0x170 [Tue Mar 17 17:58:50 2020][8592415.410239] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:50 2020][8592415.416179] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:50 2020][8592415.421496] [] ? yama_task_prctl+0x24/0x110 [Tue Mar 17 17:58:51 2020][8592415.427523] [] ? strlcpy+0x42/0x60 [Tue Mar 17 17:58:51 2020][8592415.432752] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592415.438072] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.444265] polkitd S ffff9c8a163ca080 0 50238 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.451536] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.454168] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.459315] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:51 2020][8592415.466557] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:51 2020][8592415.472494] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:51 2020][8592415.478330] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:51 2020][8592415.484880] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:51 2020][8592415.491149] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:51 2020][8592415.496730] [] ? __alloc_pages_nodemask+0x176/0x420 [Tue Mar 17 17:58:51 2020][8592415.503451] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:51 2020][8592415.508940] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:51 2020][8592415.515378] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:51 2020][8592415.521399] [] ? wake_up_q+0x5b/0x80 [Tue Mar 17 17:58:51 2020][8592415.526803] [] ? futex_wake+0x90/0x180 [Tue Mar 17 17:58:51 2020][8592415.532384] [] ? eventfd_ctx_read+0x67/0x1e0 [Tue Mar 17 17:58:51 2020][8592415.538499] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:51 2020][8592415.543728] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 17:58:51 2020][8592415.550255] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.556446] smartd S ffff9c6a23658000 0 50173 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.563723] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.566349] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:51 2020][8592415.573056] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.578211] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:51 2020][8592415.583699] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:51 2020][8592415.589706] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:51 2020][8592415.595634] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:51 2020][8592415.601302] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:51 2020][8592415.606896] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.613081] dsm_sa_eventmgr S ffff9c6a1db8b0c0 0 50187 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.620358] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.622998] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.628141] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:51 2020][8592415.635374] [] ? tlb_finish_mmu+0x55/0x60 [Tue Mar 17 17:58:51 2020][8592415.641215] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:51 2020][8592415.647313] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:51 2020][8592415.653840] [] do_sigtimedwait+0x196/0x240 [Tue Mar 17 17:58:51 2020][8592415.659770] [] SYSC_rt_sigtimedwait+0xb4/0x120 [Tue Mar 17 17:58:51 2020][8592415.666055] [] ? __do_page_fault+0x228/0x4f0 [Tue Mar 17 17:58:51 2020][8592415.672151] [] SyS_rt_sigtimedwait+0xe/0x10 [Tue Mar 17 17:58:51 2020][8592415.678173] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.684355] dsm_sa_eventmgr S ffff9c7a09e4a080 0 50228 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.691624] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.694255] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:51 2020][8592415.700961] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.706109] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:51 2020][8592415.711604] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:51 2020][8592415.717618] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:51 2020][8592415.723546] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:51 2020][8592415.729214] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:51 2020][8592415.734788] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.740973] mdadm S ffff9c8a1e310000 0 50192 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.748242] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.750873] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:51 2020][8592415.757580] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.762724] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:51 2020][8592415.769861] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:51 2020][8592415.775788] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:51 2020][8592415.783094] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:51 2020][8592415.789623] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:51 2020][8592415.795897] [] do_select+0x6d1/0x7c0 [Tue Mar 17 17:58:51 2020][8592415.801306] [] ? zone_statistics+0x88/0xa0 [Tue Mar 17 17:58:51 2020][8592415.807232] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:51 2020][8592415.812730] [] ? __alloc_pages_nodemask+0x176/0x420 [Tue Mar 17 17:58:51 2020][8592415.819434] [] ? __d_lookup+0x7a/0x160 [Tue Mar 17 17:58:51 2020][8592415.825023] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:58:51 2020][8592415.830526] [] ? kmem_cache_alloc_trace+0x3c/0x200 [Tue Mar 17 17:58:51 2020][8592415.837141] [] ? ima_get_action+0x23/0x30 [Tue Mar 17 17:58:51 2020][8592415.842994] [] ? process_measurement+0x8e/0x250 [Tue Mar 17 17:58:51 2020][8592415.849353] [] ? string.isra.7+0x3b/0xf0 [Tue Mar 17 17:58:51 2020][8592415.855119] [] ? vsnprintf+0x201/0x6a0 [Tue Mar 17 17:58:51 2020][8592415.860697] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 17:58:51 2020][8592415.866624] [] ? iput+0x3b/0x190 [Tue Mar 17 17:58:51 2020][8592415.871695] [] ? __blkdev_put+0x159/0x1c0 [Tue Mar 17 17:58:51 2020][8592415.877529] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:51 2020][8592415.883371] [] SyS_select+0xca/0x120 [Tue Mar 17 17:58:51 2020][8592415.888777] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.894956] irqbalance S ffff9c7a25b08000 0 50200 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.902224] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.904869] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:51 2020][8592415.911571] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.916739] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:51 2020][8592415.922233] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:51 2020][8592415.928250] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:51 2020][8592415.934174] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:51 2020][8592415.939840] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:51 2020][8592415.945429] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592415.951610] gssproxy S ffff9c6a24f70000 0 50203 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592415.958880] Call Trace: [Tue Mar 17 17:58:51 2020][8592415.961508] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592415.966662] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:51 2020][8592415.973885] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:58:51 2020][8592415.979900] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 17:58:51 2020][8592415.985930] [] ? cpumask_next_and+0x35/0x50 [Tue Mar 17 17:58:51 2020][8592415.991939] [] ? security_inode_alloc+0x24/0x30 [Tue Mar 17 17:58:51 2020][8592415.998297] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:51 2020][8592416.004826] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:51 2020][8592416.010060] [] ? SyS_epoll_ctl+0x130/0xc80 [Tue Mar 17 17:58:51 2020][8592416.016002] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:51 2020][8592416.021751] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:51 2020][8592416.027496] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.033676] gssproxy S ffff9c7a2b526180 0 50217 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.040944] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.043572] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.048714] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:51 2020][8592416.054931] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:51 2020][8592416.060422] [] ? handle_pte_fault+0x2f4/0xd10 [Tue Mar 17 17:58:51 2020][8592416.066611] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:51 2020][8592416.072537] [] ? idle_balance+0x7f/0x250 [Tue Mar 17 17:58:51 2020][8592416.078283] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:51 2020][8592416.083611] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592416.088926] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.095106] gssproxy S ffff9c7a2b5230c0 0 50218 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.102383] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.105011] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.110150] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:51 2020][8592416.116360] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:51 2020][8592416.121854] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:51 2020][8592416.127360] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:51 2020][8592416.133282] [] ? idle_balance+0x22d/0x250 [Tue Mar 17 17:58:51 2020][8592416.139116] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:51 2020][8592416.144451] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592416.149767] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.155946] gssproxy S ffff9c7a2b521040 0 50219 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.163214] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.165844] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.170991] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:51 2020][8592416.177186] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:51 2020][8592416.182673] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:51 2020][8592416.188939] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:51 2020][8592416.194860] [] ? idle_balance+0x7f/0x250 [Tue Mar 17 17:58:51 2020][8592416.200613] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:51 2020][8592416.205940] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592416.211257] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.217435] gssproxy S ffff9c7a2b520000 0 50220 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.224725] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.227361] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.232508] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:51 2020][8592416.238695] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:51 2020][8592416.244181] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:51 2020][8592416.250462] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:51 2020][8592416.256383] [] ? idle_balance+0x7f/0x250 [Tue Mar 17 17:58:51 2020][8592416.262129] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:51 2020][8592416.267444] [] ? schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.272779] [] ? do_exit+0x6bb/0xa40 [Tue Mar 17 17:58:51 2020][8592416.278196] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592416.283511] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.289690] gssproxy S ffff9c7a2b524100 0 50221 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.296960] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.299586] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.304732] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:51 2020][8592416.310930] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:51 2020][8592416.316418] [] ? sched_clock+0x9/0x10 [Tue Mar 17 17:58:51 2020][8592416.321919] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:51 2020][8592416.327841] [] ? idle_balance+0x22d/0x250 [Tue Mar 17 17:58:51 2020][8592416.333674] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:51 2020][8592416.339014] [] ? schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.344333] [] ? do_exit+0x6bb/0xa40 [Tue Mar 17 17:58:51 2020][8592416.349733] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:51 2020][8592416.355047] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.361235] dbus-daemon S ffff9c7a25b09040 0 50204 1 0x00000080 [Tue Mar 17 17:58:51 2020][8592416.368504] Call Trace: [Tue Mar 17 17:58:51 2020][8592416.371131] [] schedule+0x29/0x70 [Tue Mar 17 17:58:51 2020][8592416.376272] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:51 2020][8592416.383502] [] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0 [Tue Mar 17 17:58:51 2020][8592416.390482] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:51 2020][8592416.397011] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:51 2020][8592416.402236] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:51 2020][8592416.407998] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:51 2020][8592416.413748] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:51 2020][8592416.419934] chronyd S ffff9c5a368fd140 0 50237 1 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.427203] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.429833] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:52 2020][8592416.436541] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.441703] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:52 2020][8592416.448840] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:52 2020][8592416.454780] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:52 2020][8592416.462091] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.468625] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:52 2020][8592416.474898] [] do_select+0x6d1/0x7c0 [Tue Mar 17 17:58:52 2020][8592416.480302] [] ? udp_rmem_release+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592416.486326] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.491816] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.497301] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.502789] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.508300] [] ? sock_sendmsg+0xb6/0xf0 [Tue Mar 17 17:58:52 2020][8592416.513975] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 17:58:52 2020][8592416.519637] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 17:58:52 2020][8592416.525578] [] ? kmem_cache_free+0x1e2/0x200 [Tue Mar 17 17:58:52 2020][8592416.531675] [] ? __d_free+0x35/0x40 [Tue Mar 17 17:58:52 2020][8592416.536988] [] ? d_free+0x60/0x70 [Tue Mar 17 17:58:52 2020][8592416.542136] [] ? __dentry_kill+0x128/0x180 [Tue Mar 17 17:58:52 2020][8592416.548066] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:52 2020][8592416.553919] [] SyS_select+0xca/0x120 [Tue Mar 17 17:58:52 2020][8592416.559320] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592416.565501] rpcbind S ffff9c7a30aab0c0 0 50250 1 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.572805] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.575434] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:52 2020][8592416.582141] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.587290] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:52 2020][8592416.594430] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:52 2020][8592416.600350] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:52 2020][8592416.607665] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.614199] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:52 2020][8592416.620481] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:52 2020][8592416.626069] [] ? enqueue_task_fair+0x208/0x6c0 [Tue Mar 17 17:58:52 2020][8592416.632339] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:58:52 2020][8592416.638272] [] ? task_scan_min+0x3e/0x60 [Tue Mar 17 17:58:52 2020][8592416.644021] [] ? should_numa_migrate_memory+0x5b/0x150 [Tue Mar 17 17:58:52 2020][8592416.650980] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.656467] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.661951] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.667454] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.672941] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.678426] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.683941] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592416.689436] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:52 2020][8592416.695277] [] SyS_poll+0x74/0x110 [Tue Mar 17 17:58:52 2020][8592416.700526] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592416.706708] sssd_be S ffff9c8a2538a080 0 50264 50166 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.713993] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.716623] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:52 2020][8592416.723330] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.728468] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:52 2020][8592416.735603] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:52 2020][8592416.741535] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:52 2020][8592416.748846] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.755373] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:52 2020][8592416.760620] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:52 2020][8592416.766716] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:52 2020][8592416.772463] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:52 2020][8592416.778232] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592416.784413] sssd_nss S ffff9c8a25389040 0 50266 50166 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.791682] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.794311] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:52 2020][8592416.801011] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.806185] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:52 2020][8592416.813328] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:52 2020][8592416.819254] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:52 2020][8592416.826574] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.833104] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:52 2020][8592416.838337] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:52 2020][8592416.844429] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:52 2020][8592416.850175] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:52 2020][8592416.855936] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592416.862135] sssd_pam S ffff9c8a2538e180 0 50267 50166 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.869406] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.872048] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:52 2020][8592416.878752] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.883914] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:52 2020][8592416.891050] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:52 2020][8592416.896994] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:52 2020][8592416.904301] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.910836] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:52 2020][8592416.916085] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:52 2020][8592416.922194] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:52 2020][8592416.927943] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:52 2020][8592416.933704] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592416.939885] systemd-logind S ffff9c8a163c8000 0 50269 1 0x00000080 [Tue Mar 17 17:58:52 2020][8592416.947155] Call Trace: [Tue Mar 17 17:58:52 2020][8592416.949798] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592416.954940] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:52 2020][8592416.962176] [] ? ep_scan_ready_list.isra.7+0x1b9/0x1f0 [Tue Mar 17 17:58:52 2020][8592416.969137] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592416.975685] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:52 2020][8592416.980914] [] ? SyS_epoll_ctl+0x24d/0xc80 [Tue Mar 17 17:58:52 2020][8592416.986860] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:52 2020][8592416.992605] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:52 2020][8592416.998352] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:52 2020][8592417.004545] mlx4 S ffff9c7a322f5140 0 50359 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.011822] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.014454] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.019595] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.025452] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.031634] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.036693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.042974] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.049590] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.055855] mlx4_ib S ffff9c7a322f4100 0 50367 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.063144] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.065770] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.070924] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.076759] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.082939] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.088006] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.094275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.100902] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.107171] mlx4_ib_mcg S ffff9c7a322f2080 0 50368 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.114443] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.117092] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.122234] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.128080] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.134280] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.139333] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.145616] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.152245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.158512] mlx5e S ffff9c7a322f30c0 0 50419 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.165802] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.168428] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.173567] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.179422] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.185606] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.190668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.196958] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.203573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.209847] ipoib_wq S ffff9c7a322f6180 0 50420 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.217153] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.219786] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.224925] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.230776] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.236956] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.242018] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.248306] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.254921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.261188] rdma_cm S ffff9c6a23290000 0 50439 2 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.268496] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.271128] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.276268] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:52 2020][8592417.282116] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:52 2020][8592417.288297] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:52 2020][8592417.293351] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.299639] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:52 2020][8592417.306253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:52 2020][8592417.312520] sshd S ffff9c7a14791040 0 50881 1 0x00000080 [Tue Mar 17 17:58:52 2020][8592417.319807] Call Trace: [Tue Mar 17 17:58:52 2020][8592417.322436] [] schedule+0x29/0x70 [Tue Mar 17 17:58:52 2020][8592417.327584] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 17:58:52 2020][8592417.334838] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 17:58:52 2020][8592417.340680] [] ? __pollwait+0x7f/0xf0 [Tue Mar 17 17:58:52 2020][8592417.346173] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:52 2020][8592417.352715] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:52 2020][8592417.358983] [] do_select+0x6d1/0x7c0 [Tue Mar 17 17:58:52 2020][8592417.364396] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592417.369906] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:52 2020][8592417.375399] [] ? pagevec_lru_move_fn+0x10c/0x130 [Tue Mar 17 17:58:52 2020][8592417.381872] [] ? lru_deactivate_file_fn+0x1d0/0x1d0 [Tue Mar 17 17:58:52 2020][8592417.388580] [] ? __lru_cache_add+0x64/0x90 [Tue Mar 17 17:58:52 2020][8592417.394524] [] ? lru_cache_add+0xe/0x10 [Tue Mar 17 17:58:52 2020][8592417.400187] [] ? page_add_new_anon_rmap+0xb8/0x170 [Tue Mar 17 17:58:52 2020][8592417.406805] [] ? wp_page_copy.isra.71+0x3ab/0x560 [Tue Mar 17 17:58:52 2020][8592417.413345] [] ? unlock_page+0x2b/0x30 [Tue Mar 17 17:58:52 2020][8592417.418920] [] ? do_wp_page+0x19e/0x720 [Tue Mar 17 17:58:53 2020][8592417.424581] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 17:58:53 2020][8592417.430516] [] ? handle_mm_fault+0x39d/0x9b0 [Tue Mar 17 17:58:53 2020][8592417.436612] [] SyS_select+0xca/0x120 [Tue Mar 17 17:58:53 2020][8592417.442020] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.448236] rsyslogd S ffff9c7a147930c0 0 50882 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.455510] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.458139] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592417.464839] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.470000] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:53 2020][8592417.477138] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592417.483064] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:53 2020][8592417.490385] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:53 2020][8592417.496913] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:53 2020][8592417.503181] [] do_select+0x6d1/0x7c0 [Tue Mar 17 17:58:53 2020][8592417.508620] [] ? smp_call_function_many+0x1df/0x270 [Tue Mar 17 17:58:53 2020][8592417.515323] [] ? native_flush_tlb_others+0xb8/0xc0 [Tue Mar 17 17:58:53 2020][8592417.521944] [] ? __inc_zone_page_state+0x32/0x60 [Tue Mar 17 17:58:53 2020][8592417.528407] [] ? do_page_add_anon_rmap+0x9a/0xc0 [Tue Mar 17 17:58:53 2020][8592417.534848] [] ? task_scan_min+0x3e/0x60 [Tue Mar 17 17:58:53 2020][8592417.540595] [] ? task_scan_max+0x27/0x40 [Tue Mar 17 17:58:53 2020][8592417.546364] [] ? task_numa_fault+0x8d0/0xbb0 [Tue Mar 17 17:58:53 2020][8592417.552460] [] ? do_numa_page+0x1be/0x250 [Tue Mar 17 17:58:53 2020][8592417.558300] [] ? handle_pte_fault+0x316/0xd10 [Tue Mar 17 17:58:53 2020][8592417.564507] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:58:53 2020][8592417.569828] [] ? should_numa_migrate_memory+0x5b/0x150 [Tue Mar 17 17:58:53 2020][8592417.576795] [] ? mpol_misplaced+0x18c/0x1e0 [Tue Mar 17 17:58:53 2020][8592417.582816] [] ? page_add_file_rmap+0x8c/0xc0 [Tue Mar 17 17:58:53 2020][8592417.589000] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 17:58:53 2020][8592417.594924] [] ? handle_mm_fault+0x39d/0x9b0 [Tue Mar 17 17:58:53 2020][8592417.601035] [] ? __check_object_size+0x1ca/0x250 [Tue Mar 17 17:58:53 2020][8592417.607477] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:53 2020][8592417.613316] [] SyS_select+0xca/0x120 [Tue Mar 17 17:58:53 2020][8592417.618737] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.624935] in:imjournal S ffff9c7a09e4d140 0 50889 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.632206] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.634852] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592417.641552] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.646708] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:53 2020][8592417.653844] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592417.659769] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:53 2020][8592417.667090] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:53 2020][8592417.673621] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 17:58:53 2020][8592417.679910] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 17:58:53 2020][8592417.685518] [] ? jbd2_journal_dirty_metadata+0xd1/0x260 [jbd2] [Tue Mar 17 17:58:53 2020][8592417.693174] [] ? security_inode_permission+0x22/0x30 [Tue Mar 17 17:58:53 2020][8592417.699983] [] ? __inode_permission+0x52/0xd0 [Tue Mar 17 17:58:53 2020][8592417.706162] [] ? inode_permission+0x18/0x50 [Tue Mar 17 17:58:53 2020][8592417.712169] [] ? link_path_walk+0x81/0x8b0 [Tue Mar 17 17:58:53 2020][8592417.718105] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 17:58:53 2020][8592417.723593] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:53 2020][8592417.730041] [] ? kmem_cache_alloc+0x35/0x1f0 [Tue Mar 17 17:58:53 2020][8592417.736157] [] ? wake_up_q+0x5b/0x80 [Tue Mar 17 17:58:53 2020][8592417.741558] [] ? futex_wake_op+0x4a5/0x610 [Tue Mar 17 17:58:53 2020][8592417.747479] [] ? putname+0x3d/0x60 [Tue Mar 17 17:58:53 2020][8592417.752741] [] ? do_futex+0x17a/0x5a0 [Tue Mar 17 17:58:53 2020][8592417.758234] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:58:53 2020][8592417.764074] [] SyS_ppoll+0x1d3/0x1f0 [Tue Mar 17 17:58:53 2020][8592417.769489] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.775671] rs:main Q:Reg S ffff9c5a153f2080 0 50893 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.782939] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.785566] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.790720] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592417.796904] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592417.802398] [] ? generic_file_aio_write+0x77/0xa0 [Tue Mar 17 17:58:53 2020][8592417.808948] [] ? ext4_file_write+0xd2/0x1e0 [ext4] [Tue Mar 17 17:58:53 2020][8592417.815562] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592417.820875] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592417.826205] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.832385] collectd S ffff9c6a1db8a080 0 50885 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.839653] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.842298] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592417.849000] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.854161] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:53 2020][8592417.859666] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:53 2020][8592417.865677] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592417.871616] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:53 2020][8592417.877280] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:53 2020][8592417.882859] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.889039] writer#0 S ffff9c7a09e49040 0 50894 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.896324] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.898953] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.904095] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592417.910296] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592417.915799] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:53 2020][8592417.922257] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:53 2020][8592417.928266] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:53 2020][8592417.934193] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592417.939521] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592417.944836] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592417.951015] writer#1 S ffff9c7a09e4c100 0 50895 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592417.958301] Call Trace: [Tue Mar 17 17:58:53 2020][8592417.960931] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592417.966080] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592417.972294] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592417.977787] [] ? ttwu_do_wakeup+0xb5/0xe0 [Tue Mar 17 17:58:53 2020][8592417.983621] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592417.988957] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592417.994274] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.000475] writer#2 S ffff9c7a09e4b0c0 0 50896 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.007746] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.010389] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.015531] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.021733] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.027237] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:53 2020][8592418.033679] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:53 2020][8592418.039702] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:53 2020][8592418.045621] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.050935] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.056260] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.062444] writer#3 S ffff9c7a09e4e180 0 50897 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.069721] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.072365] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.077506] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.083713] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.089207] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:53 2020][8592418.095645] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:53 2020][8592418.101665] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:53 2020][8592418.107589] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.112903] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.118238] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.124419] writer#4 S ffff9c7a2b522080 0 50898 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.131688] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.134332] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.139487] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.145699] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.151191] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 17:58:53 2020][8592418.157643] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 17:58:53 2020][8592418.163653] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 17:58:53 2020][8592418.169590] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.174903] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.180216] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.186408] reader#0 S ffff9c7a3268b0c0 0 50899 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.193680] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.196312] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592418.203018] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.208170] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.214354] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.219840] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592418.225774] [] ? futex_wait_queue_me+0xa4/0x130 [Tue Mar 17 17:58:53 2020][8592418.232131] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.237450] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.242776] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.248960] reader#1 S ffff9c7a324ca080 0 50900 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.256228] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.258885] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592418.265594] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.270753] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.276937] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.282431] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592418.288365] [] ? futex_wait_queue_me+0xa4/0x130 [Tue Mar 17 17:58:53 2020][8592418.294722] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.300057] [] ? do_munmap+0x327/0x480 [Tue Mar 17 17:58:53 2020][8592418.305633] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.310954] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.317154] reader#2 S ffff9c7a324cb0c0 0 50901 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.324427] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.327056] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592418.333755] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.338911] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.345093] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.350586] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:53 2020][8592418.356521] [] ? futex_wait_queue_me+0xa4/0x130 [Tue Mar 17 17:58:53 2020][8592418.362878] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:53 2020][8592418.368198] [] ? mntput+0x24/0x40 [Tue Mar 17 17:58:53 2020][8592418.373361] [] ? __fput+0x186/0x260 [Tue Mar 17 17:58:53 2020][8592418.378675] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:53 2020][8592418.383990] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:53 2020][8592418.390189] reader#3 S ffff9c7a324cd140 0 50902 1 0x00000080 [Tue Mar 17 17:58:53 2020][8592418.397462] Call Trace: [Tue Mar 17 17:58:53 2020][8592418.400103] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:53 2020][8592418.406810] [] schedule+0x29/0x70 [Tue Mar 17 17:58:53 2020][8592418.411970] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:53 2020][8592418.418152] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:53 2020][8592418.423666] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:54 2020][8592418.429592] [] ? futex_wait_queue_me+0xa4/0x130 [Tue Mar 17 17:58:54 2020][8592418.435946] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:54 2020][8592418.441280] [] ? do_munmap+0x327/0x480 [Tue Mar 17 17:58:54 2020][8592418.446858] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:54 2020][8592418.452179] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.458380] reader#4 S ffff9c7a2f4e6180 0 50903 1 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.465651] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.468281] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.473427] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 17:58:54 2020][8592418.479634] [] futex_wait+0x17b/0x280 [Tue Mar 17 17:58:54 2020][8592418.485122] [] do_futex+0x106/0x5a0 [Tue Mar 17 17:58:54 2020][8592418.490442] [] SyS_futex+0x80/0x190 [Tue Mar 17 17:58:54 2020][8592418.495777] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.501959] nfsiod S ffff9c6a2fe76180 0 50940 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.509234] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.511883] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.517023] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:58:54 2020][8592418.522877] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:58:54 2020][8592418.529061] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592418.534142] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592418.540413] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592418.547025] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592418.553308] nfsv4.1-svc S ffff9c6a36d12080 0 50968 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.560579] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.563209] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.568364] [] nfs41_callback_svc+0x15d/0x1c0 [nfsv4] [Tue Mar 17 17:58:54 2020][8592418.575253] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:54 2020][8592418.581267] [] ? nfs4_callback_svc+0x60/0x60 [nfsv4] [Tue Mar 17 17:58:54 2020][8592418.588069] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592418.593141] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592418.599410] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592418.606045] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592418.612312] crond S ffff9c8a21be8000 0 50991 1 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.619581] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.622226] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:54 2020][8592418.628928] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.634089] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 17:58:54 2020][8592418.639579] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 17:58:54 2020][8592418.645594] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:54 2020][8592418.651547] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 17:58:54 2020][8592418.657206] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 17:58:54 2020][8592418.662780] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.668981] agetty S ffff9c6a39cdb0c0 0 51051 1 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.676254] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.678883] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.684031] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:58:54 2020][8592418.690060] [] ? get_work_pool+0x3e/0x70 [Tue Mar 17 17:58:54 2020][8592418.695810] [] ? flush_work+0x13b/0x190 [Tue Mar 17 17:58:54 2020][8592418.701480] [] ? serial8250_start_tx+0xa5/0xe0 [Tue Mar 17 17:58:54 2020][8592418.707780] [] n_tty_read+0x2b0/0xca0 [Tue Mar 17 17:58:54 2020][8592418.713274] [] ? list_del+0xd/0x30 [Tue Mar 17 17:58:54 2020][8592418.718506] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592418.724268] [] tty_read+0x90/0x100 [Tue Mar 17 17:58:54 2020][8592418.729498] [] vfs_read+0x9f/0x170 [Tue Mar 17 17:58:54 2020][8592418.734732] [] SyS_read+0x7f/0xf0 [Tue Mar 17 17:58:54 2020][8592418.739892] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.746077] agetty S ffff9c6a39cdd140 0 51054 1 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.753351] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.755995] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.761139] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:58:54 2020][8592418.767181] [] ? flush_work+0x3a/0x190 [Tue Mar 17 17:58:54 2020][8592418.772760] [] n_tty_read+0x2b0/0xca0 [Tue Mar 17 17:58:54 2020][8592418.778258] [] ? console_unlock+0x216/0x4b0 [Tue Mar 17 17:58:54 2020][8592418.784269] [] ? list_del+0xd/0x30 [Tue Mar 17 17:58:54 2020][8592418.789495] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592418.795240] [] tty_read+0x90/0x100 [Tue Mar 17 17:58:54 2020][8592418.800468] [] vfs_read+0x9f/0x170 [Tue Mar 17 17:58:54 2020][8592418.805691] [] SyS_read+0x7f/0xf0 [Tue Mar 17 17:58:54 2020][8592418.810832] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.817026] master S ffff9c5a1c180000 0 51202 1 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.824299] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.826927] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:54 2020][8592418.833626] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.838765] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:54 2020][8592418.845902] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:54 2020][8592418.851826] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:54 2020][8592418.859132] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:54 2020][8592418.865675] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:54 2020][8592418.870901] [] ? hrtimer_start+0x12/0x20 [Tue Mar 17 17:58:54 2020][8592418.876647] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592418.882406] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:54 2020][8592418.888159] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.894346] qmgr S ffff9c8a30bb2080 0 51242 51202 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.901640] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.904269] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 17:58:54 2020][8592418.910969] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.916117] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 17:58:54 2020][8592418.923257] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 17:58:54 2020][8592418.929178] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 17:58:54 2020][8592418.936507] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 17:58:54 2020][8592418.943035] [] ep_poll+0x23e/0x360 [Tue Mar 17 17:58:54 2020][8592418.948264] [] ? mntput+0x24/0x40 [Tue Mar 17 17:58:54 2020][8592418.953424] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592418.959173] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 17:58:54 2020][8592418.964928] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 17:58:54 2020][8592418.971128] cfs_rh_00 S ffff9c5ae9670000 0 64739 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592418.978401] Call Trace: [Tue Mar 17 17:58:54 2020][8592418.981030] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592418.986190] [] cfs_wi_scheduler+0x30c/0x460 [libcfs] [Tue Mar 17 17:58:54 2020][8592418.993010] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:54 2020][8592418.999027] [] ? cfs_wi_sched_create+0x680/0x680 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.006267] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.011321] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.017597] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.024231] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.030502] cfs_rh_01 S ffff9c6a2074d140 0 64740 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.037778] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.040418] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.045568] [] cfs_wi_scheduler+0x30c/0x460 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.052388] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:54 2020][8592419.058403] [] ? cfs_wi_sched_create+0x680/0x680 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.065640] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.070697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.076984] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.083600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.089882] cfs_rh_02 S ffff9c6a00e88000 0 64741 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.097153] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.099781] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.104927] [] cfs_wi_scheduler+0x30c/0x460 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.111746] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:54 2020][8592419.117761] [] ? cfs_wi_sched_create+0x680/0x680 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.124998] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.130054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.136350] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.142967] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.149234] cfs_rh_03 S ffff9c6a00e8e180 0 64742 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.156537] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.159166] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.164320] [] cfs_wi_scheduler+0x30c/0x460 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.171138] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:54 2020][8592419.177154] [] ? cfs_wi_sched_create+0x680/0x680 [libcfs] [Tue Mar 17 17:58:54 2020][8592419.184378] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.189446] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.195714] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.202326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.208605] ptlrpc_hr00_000 S ffff9c6a00e89040 0 64748 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.215879] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.218509] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:54 2020][8592419.224609] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.229834] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.236449] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592419.242234] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.249211] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.254266] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.260533] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.267168] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.273438] ptlrpc_hr00_001 S ffff9c6a00e8b0c0 0 64749 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.280713] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.283358] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:54 2020][8592419.289456] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.294661] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.301276] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592419.307058] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.314037] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.319093] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.325361] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.331994] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.338264] ptlrpc_hr00_002 S ffff9c6a00e8a080 0 64750 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.345531] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.348174] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:54 2020][8592419.354272] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.359477] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.366093] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:54 2020][8592419.371867] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:54 2020][8592419.378848] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:54 2020][8592419.383903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.390179] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:54 2020][8592419.396815] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:54 2020][8592419.403084] ptlrpc_hr00_003 S ffff9c6a00e8d140 0 64751 2 0x00000080 [Tue Mar 17 17:58:54 2020][8592419.410377] Call Trace: [Tue Mar 17 17:58:54 2020][8592419.413021] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:54 2020][8592419.419117] [] schedule+0x29/0x70 [Tue Mar 17 17:58:54 2020][8592419.424308] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.430928] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.436717] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.443679] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.448740] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.455030] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.461643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.467909] ptlrpc_hr00_004 S ffff9c6a00e8c100 0 64752 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.475196] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.477824] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.483916] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.489101] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.495721] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.501496] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.508476] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.513532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.519806] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.526442] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.532710] ptlrpc_hr00_005 S ffff9c6a2365e180 0 64753 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.539980] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.542609] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.548725] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.553896] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.560513] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.566304] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.573272] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.578348] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.584633] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.591245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.597526] ptlrpc_hr01_000 S ffff9c6a22ab1040 0 64754 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.604797] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.607425] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.613520] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.618713] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.625333] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.631117] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.638096] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.643150] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.649445] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.656063] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.662331] ptlrpc_hr01_001 S ffff9c6a22ab30c0 0 64755 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.669626] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.672255] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.678357] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.683546] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.690160] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.695943] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.702936] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.707996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.714269] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.720898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.727166] ptlrpc_hr01_002 S ffff9c6a30a21040 0 64756 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.734433] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.737080] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.743175] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.748365] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.754987] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.760770] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.767767] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.772823] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.779098] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.785737] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.792010] ptlrpc_hr01_003 S ffff9c6a30a26180 0 64757 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.799280] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.801922] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.808020] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.813210] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.819832] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.825630] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.832597] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.837650] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.843938] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.850554] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.856821] ptlrpc_hr01_004 S ffff9c6a35a55140 0 64758 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.864113] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.866743] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.872836] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.878034] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.884648] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.890424] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.897403] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.902458] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.908745] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.915365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.921637] ptlrpc_hr01_005 S ffff9c6a35a51040 0 64759 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.928925] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.931552] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592419.937647] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592419.942852] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.949469] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592419.955267] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592419.962234] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592419.967294] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.973575] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592419.980192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592419.986466] ptlrpc_hr02_000 S ffff9c6a35a530c0 0 64760 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592419.993751] Call Trace: [Tue Mar 17 17:58:55 2020][8592419.996380] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.002473] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.007670] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.014286] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.020060] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.027042] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.032095] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.038363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.044996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.051267] ptlrpc_hr02_001 S ffff9c6a39c4e180 0 64761 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.058536] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.061190] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.067284] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.072476] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.079096] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.084888] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.091852] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.096906] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.103195] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.109810] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.116077] ptlrpc_hr02_002 S ffff9c6a39c48000 0 64762 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.123362] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.125991] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.132085] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.137276] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.143913] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.149689] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.156669] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.161726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.168013] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.174657] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.180929] ptlrpc_hr02_003 S ffff9c6a39c4c100 0 64763 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.188215] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.190845] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.196939] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.202129] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.208750] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.214539] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.221499] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.226561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.232863] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.239482] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.245748] ptlrpc_hr02_004 S ffff9c6a39c4b0c0 0 64764 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.253035] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.255664] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.261763] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.266946] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.273559] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.279334] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.286315] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.291368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.297663] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.304284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.310570] ptlrpc_hr02_005 S ffff9c6a39c4d140 0 64765 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.317845] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.320473] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.326576] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.331770] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.338388] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.344185] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.351153] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.356219] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.362489] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:55 2020][8592420.369110] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.375394] ptlrpc_hr03_000 S ffff9c6a341fd140 0 64766 2 0x00000080 [Tue Mar 17 17:58:55 2020][8592420.382662] Call Trace: [Tue Mar 17 17:58:55 2020][8592420.385290] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:55 2020][8592420.391385] [] schedule+0x29/0x70 [Tue Mar 17 17:58:55 2020][8592420.396576] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.403198] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:55 2020][8592420.408979] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:55 2020][8592420.415962] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:55 2020][8592420.421015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:55 2020][8592420.427282] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.433917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.440185] ptlrpc_hr03_001 S ffff9c6a341fb0c0 0 64767 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.447454] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.450098] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:56 2020][8592420.456195] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.461384] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.467999] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.473796] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.480756] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.485818] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.492108] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.498723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.504996] ptlrpc_hr03_002 S ffff9c6a341fe180 0 64768 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.512282] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.514910] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:56 2020][8592420.521001] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.526200] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.532817] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.538599] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.545579] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.550634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.556901] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.563538] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.569807] ptlrpc_hr03_003 S ffff9c6a24fc1040 0 64769 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.577083] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.579712] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:56 2020][8592420.585819] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.591005] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.597625] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.603415] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.610375] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.615435] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.621718] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.628332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.634598] ptlrpc_hr03_004 S ffff9c6a24fc0000 0 64770 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.641883] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.644512] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:56 2020][8592420.650605] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.655802] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.662420] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.668201] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.675183] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.680237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.686504] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.693140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.699407] ptlrpc_hr03_005 S ffff9c7a36a29040 0 64771 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.706676] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.709335] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:58:56 2020][8592420.715435] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.720625] [] ptlrpc_hr_main+0x46d/0x580 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.727247] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.733043] [] ? ptlrpc_handle_rs+0x5f0/0x5f0 [ptlrpc] [Tue Mar 17 17:58:56 2020][8592420.740003] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.745064] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.751344] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.757960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.764227] kiblnd_connd S ffff9c7a36a2c100 0 64781 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.771512] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.774142] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.779289] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:56 2020][8592420.785310] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:56 2020][8592420.791848] [] kiblnd_connd+0x60c/0xa00 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.798469] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.804233] [] ? kiblnd_cm_callback+0x2380/0x2380 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.811719] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.816772] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.823069] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.829688] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.835959] kiblnd_sd_00_00 S ffff9c7a36a2b0c0 0 64782 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.843248] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.845877] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.851028] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.858093] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592420.864363] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592420.870043] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.875798] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.882675] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.887769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.894037] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.900648] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.906916] kiblnd_sd_00_01 S ffff9c7a36a2e180 0 64783 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.914193] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.916821] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.921967] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.929016] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592420.935281] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592420.940956] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592420.946710] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.953585] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592420.958654] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.964923] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592420.971546] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592420.977833] kiblnd_sd_00_02 S ffff9c7a36a2a080 0 64784 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592420.985104] Call Trace: [Tue Mar 17 17:58:56 2020][8592420.987733] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592420.992899] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592420.999982] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.006254] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.011934] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.017688] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.024566] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.029633] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.035901] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.042516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.048804] kiblnd_sd_00_03 S ffff9c6a20f41040 0 64785 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592421.056085] Call Trace: [Tue Mar 17 17:58:56 2020][8592421.058714] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592421.063864] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.070945] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.077217] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.082884] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.088647] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.095521] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.100574] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.106861] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.113477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.119771] kiblnd_sd_01_00 S ffff9c6a20f430c0 0 64786 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592421.127048] Call Trace: [Tue Mar 17 17:58:56 2020][8592421.129677] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592421.134828] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.141910] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.148180] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.153877] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.159632] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.166510] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.171587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.177868] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.184500] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.190768] kiblnd_sd_01_01 S ffff9c6a20f42080 0 64787 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592421.198036] Call Trace: [Tue Mar 17 17:58:56 2020][8592421.200665] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592421.205825] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.212877] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.219144] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.224825] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.230580] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.237485] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.242545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.248833] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.255448] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.261740] kiblnd_sd_01_02 S ffff9c6a20f45140 0 64788 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592421.269009] Call Trace: [Tue Mar 17 17:58:56 2020][8592421.271636] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592421.276780] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.283848] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.290116] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.295782] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.301560] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.308435] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.313490] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.319778] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.326394] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.332660] kiblnd_sd_01_03 S ffff9c6a20f44100 0 64789 2 0x00000080 [Tue Mar 17 17:58:56 2020][8592421.339945] Call Trace: [Tue Mar 17 17:58:56 2020][8592421.342575] [] schedule+0x29/0x70 [Tue Mar 17 17:58:56 2020][8592421.347726] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.354815] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:56 2020][8592421.361090] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:56 2020][8592421.366754] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:56 2020][8592421.372507] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:56 2020][8592421.379396] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:56 2020][8592421.384452] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.390734] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:56 2020][8592421.397354] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:56 2020][8592421.403636] kiblnd_sd_02_00 S ffff9c6a20f40000 0 64790 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.410909] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.413539] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.418703] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.425773] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.432041] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.437716] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.443468] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.450361] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.455416] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.461689] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.468317] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.474600] kiblnd_sd_02_01 S ffff9c6a20f46180 0 64791 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.481871] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.484500] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.489658] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.496715] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.502987] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.508660] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.514412] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.521290] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.526359] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.532639] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.539255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.545537] kiblnd_sd_02_02 S ffff9c7a2de19040 0 64792 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.552809] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.555440] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.560591] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.567656] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.573923] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.579597] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.585350] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.592227] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.597307] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.603581] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.610195] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.616487] kiblnd_sd_02_03 S ffff9c7a38ac8000 0 64793 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.623764] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.626404] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.631553] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.638620] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.644886] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.650546] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.656309] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.663181] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.668235] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.674516] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.681133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.687407] kiblnd_sd_03_00 S ffff9c7a38aca080 0 64794 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.694700] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.697331] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.702482] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.709559] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.715831] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.721490] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.727255] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.734129] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.739190] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.745472] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.752086] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.758359] kiblnd_sd_03_01 S ffff9c7a38acc100 0 64795 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.765647] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.768274] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.773418] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.780498] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.786770] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.792429] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.798193] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.805066] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.810117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.816398] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.823014] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.829280] kiblnd_sd_03_02 S ffff9c7a38acb0c0 0 64796 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.836566] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.839195] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.844337] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.851403] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.857672] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.863333] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.869104] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.875977] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.881054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.887336] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.893950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.900232] kiblnd_sd_03_03 S ffff9c7a2e336180 0 64797 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.907503] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.910134] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.915283] [] kiblnd_scheduler+0x32c/0x1180 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.922349] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:58:57 2020][8592421.928619] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:58:57 2020][8592421.934286] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592421.940051] [] ? kiblnd_cq_event+0x90/0x90 [ko2iblnd] [Tue Mar 17 17:58:57 2020][8592421.946947] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592421.952000] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.958284] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592421.964898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592421.971162] monitor_thread S ffff9c7a2e330000 0 64798 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592421.978449] Call Trace: [Tue Mar 17 17:58:57 2020][8592421.981078] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592421.986218] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:57 2020][8592421.992237] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:57 2020][8592421.998767] [] ? prepare_to_wait+0x56/0x90 [Tue Mar 17 17:58:57 2020][8592422.004700] [] lnet_monitor_thread+0x492/0x580 [lnet] [Tue Mar 17 17:58:57 2020][8592422.011606] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:57 2020][8592422.017626] [] ? lnet_recover_peer_nis+0x670/0x670 [lnet] [Tue Mar 17 17:58:57 2020][8592422.024853] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.029920] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.036191] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.042811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.049093] lnet_discovery S ffff9c5a16e1e180 0 64803 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.056363] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.058993] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.064141] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:58:57 2020][8592422.070181] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:58:57 2020][8592422.076708] [] ? prepare_to_wait+0x56/0x90 [Tue Mar 17 17:58:57 2020][8592422.082645] [] lnet_peer_discovery+0xdf/0x1150 [lnet] [Tue Mar 17 17:58:57 2020][8592422.089542] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:58:57 2020][8592422.095556] [] ? lnet_peer_merge_data+0xde0/0xde0 [lnet] [Tue Mar 17 17:58:57 2020][8592422.102703] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.107759] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.114025] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.120654] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.126936] ptlrpcd_rcv S ffff9c7a30aad140 0 64804 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.134208] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.136884] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:57 2020][8592422.143606] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.148810] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.154820] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592422.160626] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.167327] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.172387] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.178669] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.185284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.191587] ptlrpcd_00_00 S ffff9c7a30aa8000 0 64805 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.198860] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.201510] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:57 2020][8592422.208216] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.213409] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.219423] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592422.225207] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.231925] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.236981] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.243256] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.249893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.256159] ptlrpcd_00_01 S ffff9c7a30aaa080 0 64806 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.263429] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.266087] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:57 2020][8592422.272792] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.277975] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.283990] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592422.289792] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.296494] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.301546] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.307843] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.314460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.320726] ptlrpcd_00_02 S ffff9c5a182e8000 0 64807 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.328013] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.330660] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:57 2020][8592422.337370] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.342559] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.348575] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592422.354390] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.361125] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.366185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.372473] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:57 2020][8592422.379089] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:57 2020][8592422.385355] ptlrpcd_00_03 S ffff9c5a182ea080 0 64808 2 0x00000080 [Tue Mar 17 17:58:57 2020][8592422.392641] Call Trace: [Tue Mar 17 17:58:57 2020][8592422.395288] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:57 2020][8592422.401996] [] schedule+0x29/0x70 [Tue Mar 17 17:58:57 2020][8592422.407190] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.413200] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:57 2020][8592422.419004] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:57 2020][8592422.425708] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:57 2020][8592422.430769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.437057] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.443675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.449950] ptlrpcd_00_04 S ffff9c5a182e9040 0 64809 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.457242] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.459889] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.466597] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.471790] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.477818] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.483595] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.490314] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.495371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.501661] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.508274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.514555] ptlrpcd_00_05 S ffff9c5a182eb0c0 0 64810 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.521827] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.524473] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.531184] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.536386] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.542396] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.548173] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.554892] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.559948] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.566222] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.572856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.579125] ptlrpcd_00_06 S ffff9c5a182ec100 0 64811 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.586393] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.589055] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.595756] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.600955] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.606963] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.612742] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.619457] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.624513] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.630780] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.637414] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.643685] ptlrpcd_00_07 S ffff9c5a182ee180 0 64812 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.650960] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.653607] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.660323] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.665505] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.671514] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.677313] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.684018] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.689073] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.695363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.701977] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.708251] ptlrpcd_00_08 S ffff9c4b6976e180 0 64813 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.715547] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.718193] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.724900] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.730084] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.736099] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.741896] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.748604] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.753665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.759945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.766562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.772838] ptlrpcd_00_09 S ffff9c5a1c186180 0 64814 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.780146] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.782794] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.789503] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.794695] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.800707] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.806498] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.813205] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.818256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.824539] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.831168] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.837438] ptlrpcd_00_10 S ffff9c5a1c181040 0 64815 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.844722] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.847370] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.854071] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.859264] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.865276] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.871059] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.877777] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.882834] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.889137] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.895757] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.902030] ptlrpcd_00_11 S ffff9c5a1c1830c0 0 64816 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.909318] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.911963] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.918672] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.923862] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.929869] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592422.935668] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.942375] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592422.947435] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.953730] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592422.960348] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592422.966616] ptlrpcd_01_00 S ffff9c5a1c185140 0 64817 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592422.973910] Call Trace: [Tue Mar 17 17:58:58 2020][8592422.976555] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592422.983263] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592422.988446] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592422.994453] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.000228] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.006961] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.012021] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.018312] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.024924] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.031191] ptlrpcd_01_01 S ffff9c5a1c182080 0 64818 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.038477] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.041122] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.047824] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.053016] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.059028] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.064832] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.071535] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.076597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.082884] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.089501] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.095788] ptlrpcd_01_02 S ffff9c5a1c184100 0 64819 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.103062] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.105706] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.112406] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.117590] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.123619] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.129397] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.136116] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.141171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.147469] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.154085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.160352] ptlrpcd_01_03 S ffff9c5a36a9c100 0 64820 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.167646] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.170291] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.177001] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.182206] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.188215] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.193988] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.200709] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.205763] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.212031] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.218668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.224936] ptlrpcd_01_04 S ffff9c5a36a9d140 0 64821 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.232205] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.234865] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.241569] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.246774] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.252781] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.258556] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.265276] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.270333] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.276608] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.283241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.289513] ptlrpcd_01_05 S ffff9c5a36a99040 0 64822 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.296787] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.299450] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.306154] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.311355] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.317367] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.323150] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.329873] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.334934] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.341209] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.347843] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.354132] ptlrpcd_01_06 S ffff9c5a36a9e180 0 64823 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.361406] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.364081] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.370789] [] schedule+0x29/0x70 [Tue Mar 17 17:58:58 2020][8592423.375981] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.381995] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:58 2020][8592423.387778] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:58 2020][8592423.394497] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:58 2020][8592423.399553] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.405821] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:58 2020][8592423.412456] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:58 2020][8592423.418736] ptlrpcd_01_07 S ffff9c5a36a9b0c0 0 64824 2 0x00000080 [Tue Mar 17 17:58:58 2020][8592423.426010] Call Trace: [Tue Mar 17 17:58:58 2020][8592423.428657] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:58 2020][8592423.435380] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.440550] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.446561] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.452359] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.459057] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.464112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.470402] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.477030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.483299] ptlrpcd_01_08 S ffff9c5a36a98000 0 64825 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.490584] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.493231] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.499930] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.505115] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.511128] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.516904] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.523622] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.528680] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.534953] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.541594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.547866] ptlrpcd_01_09 S ffff9c6a0afd0000 0 64826 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.555134] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.557795] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.564499] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.569688] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.575696] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.581485] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.588194] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.593281] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.599556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.606175] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.612459] ptlrpcd_01_10 S ffff9c6a0afd1040 0 64827 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.619728] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.622374] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.629082] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.634267] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.640281] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.646080] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.652802] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.657855] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.664136] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.670752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.677016] ptlrpcd_01_11 S ffff9c6a0afd2080 0 64828 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.684304] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.686949] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.693649] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.698832] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.704847] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.710656] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.717360] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.722413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.728694] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.735310] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.741592] ptlrpcd_02_00 S ffff9c6a0afd30c0 0 64829 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.748862] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.751507] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.758207] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.763390] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.769411] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.775190] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.781908] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.786965] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.793253] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.799868] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.806149] ptlrpcd_02_01 S ffff9c6a0afd4100 0 64830 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.813420] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.816067] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.822790] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.827976] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.833990] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.839789] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.846496] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.851548] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.857828] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.864445] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.870735] ptlrpcd_02_02 S ffff9c6a0afd5140 0 64831 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.878004] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.880655] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.887359] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.892560] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.898574] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.904359] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.911077] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.916132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.922412] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.929030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.935303] ptlrpcd_02_03 S ffff9c6a0afd6180 0 64832 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592423.942588] Call Trace: [Tue Mar 17 17:58:59 2020][8592423.945235] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592423.951945] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592423.957149] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.963158] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592423.968932] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592423.975653] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592423.980708] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.986975] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592423.993603] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592423.999872] ptlrpcd_02_04 S ffff9c6a07270000 0 64833 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.007165] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.009827] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.016527] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.021711] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.027717] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.033502] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.040220] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.045276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.051557] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.058171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.064438] ptlrpcd_02_05 S ffff9c6a07271040 0 64834 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.071723] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.074370] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.081080] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.086270] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.092285] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.098082] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.104782] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.109842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.116125] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.122758] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.129031] ptlrpcd_02_06 S ffff9c6a07272080 0 64835 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.136316] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.138965] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.145672] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.150877] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.156885] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.162662] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.169382] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.174450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.180754] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.187367] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.193648] ptlrpcd_02_07 S ffff9c6a072730c0 0 64836 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.200920] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.203564] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.210265] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.215455] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.221462] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.227261] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.233968] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.239029] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.245322] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.251941] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.258208] ptlrpcd_02_08 S ffff9c6a07274100 0 64837 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.265495] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.268140] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.274841] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.280035] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.286048] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.291828] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.298563] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.303623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.309896] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.316525] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.322795] ptlrpcd_02_09 S ffff9c6a07275140 0 64838 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.330071] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.332717] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.339441] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.344611] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.350622] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:58:59 2020][8592424.356442] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.363146] [] kthread+0xd1/0xe0 [Tue Mar 17 17:58:59 2020][8592424.368206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.374488] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:58:59 2020][8592424.381101] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:58:59 2020][8592424.387370] ptlrpcd_02_10 S ffff9c6a07276180 0 64839 2 0x00000080 [Tue Mar 17 17:58:59 2020][8592424.394663] Call Trace: [Tue Mar 17 17:58:59 2020][8592424.397311] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:58:59 2020][8592424.404017] [] schedule+0x29/0x70 [Tue Mar 17 17:58:59 2020][8592424.409204] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:58:59 2020][8592424.415229] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.421008] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.427728] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.432784] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.439056] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.445684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.451953] ptlrpcd_02_11 S ffff9c6a0a720000 0 64840 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.459229] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.461891] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.468595] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.473801] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.479808] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.485582] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.492303] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.497357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.503623] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.510251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.516520] ptlrpcd_03_00 S ffff9c6a0a721040 0 64841 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.523788] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.526450] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.533151] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.538352] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.544366] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.550143] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.556860] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.561918] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.568206] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.574820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.581103] ptlrpcd_03_01 S ffff9c6a0a722080 0 64842 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.588372] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.591020] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.597728] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.602918] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.608926] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.614725] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.621431] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.626485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.632774] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.639389] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.645663] ptlrpcd_03_02 S ffff9c6a0a7230c0 0 64843 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.652959] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.655605] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.662313] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.667506] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.673519] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.679302] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.686021] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.691078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.697353] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.703986] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.710271] ptlrpcd_03_03 S ffff9c6a0a724100 0 64844 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.717542] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.720202] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.726908] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.732098] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.738111] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.743902] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.750609] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.755670] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.761953] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.768581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.774849] ptlrpcd_03_04 S ffff9c6a0a725140 0 64845 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.782135] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.784783] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.791490] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.796682] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.802697] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.808497] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.815203] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.820262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.826559] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.833176] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.839443] ptlrpcd_03_05 S ffff9c6a0a726180 0 64846 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.846727] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.849376] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.856084] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.861276] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.867291] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.873088] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.879796] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.884855] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.891151] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.897769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.904036] ptlrpcd_03_06 S ffff9c6a02be8000 0 64847 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.911323] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.913969] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.920676] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.925869] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.931883] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592424.937665] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.944398] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592424.949459] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.955733] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592424.962361] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592424.968630] ptlrpcd_03_07 S ffff9c6a02be9040 0 64848 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592424.975906] Call Trace: [Tue Mar 17 17:59:00 2020][8592424.978552] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592424.985268] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592424.990437] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592424.996449] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592425.002251] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.008954] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.014007] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.020288] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.026905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.033179] ptlrpcd_03_08 S ffff9c6a02bea080 0 64849 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.040466] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.043110] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592425.049820] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.055011] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.061041] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592425.066817] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.073537] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.078591] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.084873] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.091489] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.097776] ptlrpcd_03_09 S ffff9c6a02beb0c0 0 64850 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.105050] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.107696] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592425.114405] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.119608] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.125620] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592425.131401] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.138122] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.143177] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.149457] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.156087] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.162356] ptlrpcd_03_10 S ffff9c6a02bec100 0 64851 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.169651] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.172297] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592425.179005] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.184204] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.190211] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592425.195986] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.202708] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.207762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.214037] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.220672] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.226941] ptlrpcd_03_11 S ffff9c6a02bed140 0 64852 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.234208] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.236869] [] ? lu_env_refill+0x24/0x30 [obdclass] [Tue Mar 17 17:59:00 2020][8592425.243574] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.248781] [] ptlrpcd+0x2ee/0x550 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.254797] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:00 2020][8592425.260593] [] ? ptlrpcd_check+0x590/0x590 [ptlrpc] [Tue Mar 17 17:59:00 2020][8592425.267292] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.272360] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.278630] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.285252] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.291540] raid5wq S ffff9c8a30bb5140 0 65596 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.298817] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.301477] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.306627] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:00 2020][8592425.312480] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:00 2020][8592425.318661] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.323713] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.329994] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.336611] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.342883] bioset S ffff9c7a31c9d140 0 65597 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.350173] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.352807] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.357948] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:00 2020][8592425.363817] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:00 2020][8592425.370002] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:00 2020][8592425.375057] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.381344] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:00 2020][8592425.387961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:00 2020][8592425.394250] bioset S ffff9c7a31c9e180 0 65598 2 0x00000080 [Tue Mar 17 17:59:00 2020][8592425.401524] Call Trace: [Tue Mar 17 17:59:00 2020][8592425.404159] [] schedule+0x29/0x70 [Tue Mar 17 17:59:00 2020][8592425.409307] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:00 2020][8592425.415164] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.421344] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.426405] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.432686] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.439302] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.445577] md6_raid6 D ffff9c5a162c30c0 0 65600 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.452863] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.455489] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.460633] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:01 2020][8592425.467885] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.473903] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:01 2020][8592425.479307] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:01 2020][8592425.485426] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 17:59:01 2020][8592425.491345] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 17:59:01 2020][8592425.497526] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:01 2020][8592425.504074] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:01 2020][8592425.509475] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.515482] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:01 2020][8592425.520895] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.525965] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.532235] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.538863] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.545130] bioset S ffff9c5a162c6180 0 65623 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.552402] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.555042] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.560186] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592425.566041] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.572223] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.577283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.583579] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.590199] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.596473] bioset S ffff9c5a162c4100 0 65624 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.603771] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.606405] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.611553] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592425.617408] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.623591] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.628643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.634924] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.641555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.647823] md10_raid6 D ffff9c6a37fe4100 0 65692 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.655109] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.657738] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.662885] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:01 2020][8592425.670119] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.676127] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:01 2020][8592425.681528] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:01 2020][8592425.687636] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 17:59:01 2020][8592425.693557] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 17:59:01 2020][8592425.699736] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:01 2020][8592425.706290] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:01 2020][8592425.711696] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.717703] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:01 2020][8592425.723126] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.728181] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.734448] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.741084] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.747350] bioset S ffff9c8a04e91040 0 65718 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.754623] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.757271] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.762413] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592425.768275] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.774462] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.779522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.785804] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.792419] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.798693] bioset S ffff9c8a04e92080 0 65719 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.805983] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.808617] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.813764] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592425.819612] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.825795] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.830856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.837146] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.843760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.850025] md4_raid6 D ffff9c8a04e94100 0 65845 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.857311] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.859942] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.865089] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:01 2020][8592425.872336] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.878351] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:01 2020][8592425.883758] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:01 2020][8592425.889875] [] raid5d+0x48a/0x670 [raid456] [Tue Mar 17 17:59:01 2020][8592425.895884] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:01 2020][8592425.901289] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592425.907324] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:01 2020][8592425.912731] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.917791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.924072] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.930701] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.936970] bioset S ffff9c6a0225d140 0 66007 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.944261] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.946895] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592425.952043] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592425.957898] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592425.964079] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592425.969135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.975436] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592425.982054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592425.988321] bioset S ffff9c6a0225b0c0 0 66010 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592425.995609] Call Trace: [Tue Mar 17 17:59:01 2020][8592425.998236] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.003374] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592426.009223] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592426.015404] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.020457] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.026740] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.033354] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.039648] md8_raid6 D ffff9c6a02684100 0 66071 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.046923] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.049552] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.054692] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:01 2020][8592426.061933] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592426.067945] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:01 2020][8592426.073351] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:01 2020][8592426.079468] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 17:59:01 2020][8592426.085389] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 17:59:01 2020][8592426.091568] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:01 2020][8592426.098118] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:01 2020][8592426.103520] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592426.109534] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:01 2020][8592426.114950] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.120003] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.126269] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.132896] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.139169] bioset S ffff9c5a1139a080 0 66113 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.146447] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.149097] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.154255] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592426.160124] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592426.166323] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.171392] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.177662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.184291] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.190560] bioset S ffff9c5a1139b0c0 0 66114 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.197840] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.200478] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.205623] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592426.211480] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592426.217662] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.222721] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.229000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.235619] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.241892] md0_raid6 D ffff9c5a1139c100 0 66171 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.249177] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.251807] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.256948] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:01 2020][8592426.264203] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592426.270214] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:01 2020][8592426.275616] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:01 2020][8592426.281733] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 17:59:01 2020][8592426.287654] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 17:59:01 2020][8592426.293842] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:01 2020][8592426.300390] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:01 2020][8592426.305793] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:01 2020][8592426.311806] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:01 2020][8592426.317220] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.322292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.328561] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.335193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.341465] bioset S ffff9c7a31c9a080 0 66217 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.348736] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.351385] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.356529] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592426.362385] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592426.368564] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:01 2020][8592426.373618] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.379908] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:01 2020][8592426.386522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:01 2020][8592426.392789] bioset S ffff9c6a02bee180 0 66218 2 0x00000080 [Tue Mar 17 17:59:01 2020][8592426.400078] Call Trace: [Tue Mar 17 17:59:01 2020][8592426.402710] [] schedule+0x29/0x70 [Tue Mar 17 17:59:01 2020][8592426.407852] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:01 2020][8592426.413708] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:01 2020][8592426.419890] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.424950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.431232] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.437862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.444131] md2_raid6 D ffff9c5a3a3d9040 0 66282 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.451426] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.454055] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.459202] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 17:59:02 2020][8592426.466435] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592426.472445] [] set_in_sync+0x67/0xe0 [Tue Mar 17 17:59:02 2020][8592426.477853] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 17:59:02 2020][8592426.483960] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 17:59:02 2020][8592426.489882] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 17:59:02 2020][8592426.496060] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:02 2020][8592426.502616] [] md_thread+0x16d/0x1e0 [Tue Mar 17 17:59:02 2020][8592426.508023] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592426.514036] [] ? find_pers+0x80/0x80 [Tue Mar 17 17:59:02 2020][8592426.519447] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.524505] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.530771] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.537398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.543667] kmmpd-md6 D ffff9c6a24f75140 0 66363 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.550936] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.553565] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.558892] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.564036] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:02 2020][8592426.570050] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:02 2020][8592426.576246] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:02 2020][8592426.582078] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.587407] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:02 2020][8592426.593587] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:02 2020][8592426.598989] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:02 2020][8592426.604411] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:02 2020][8592426.609999] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.615316] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:02 2020][8592426.621778] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.627876] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:02 2020][8592426.633727] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.640542] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.646473] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.653196] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.658249] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.664530] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.671160] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.677429] jbd2/md6-8 D ffff9c6a24f76180 0 66371 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.684715] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.687345] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.692501] [] jbd2_journal_commit_transaction+0x23c/0x19b0 [jbd2] [Tue Mar 17 17:59:02 2020][8592426.700517] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:02 2020][8592426.706787] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:02 2020][8592426.712466] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592426.718474] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:02 2020][8592426.724135] [] ? try_to_del_timer_sync+0x5e/0x90 [Tue Mar 17 17:59:02 2020][8592426.730608] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:02 2020][8592426.736614] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592426.742625] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:02 2020][8592426.749084] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.754140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.760429] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.767043] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.773308] ldiskfs-rsv-con S ffff9c6a02681040 0 66499 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.780600] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.783236] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.788383] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:02 2020][8592426.794250] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:02 2020][8592426.800437] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.805491] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.811777] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.818395] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.824685] kmmpd-md10 D ffff9c6a02685140 0 66556 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.831954] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.834584] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.839896] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.845065] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:02 2020][8592426.851078] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:02 2020][8592426.857263] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:02 2020][8592426.863111] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.868427] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:02 2020][8592426.874608] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:02 2020][8592426.880030] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:02 2020][8592426.885431] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:02 2020][8592426.891004] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592426.896331] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:02 2020][8592426.902788] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.908882] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:02 2020][8592426.914737] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.921537] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.927464] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592426.934178] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592426.939235] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.945510] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592426.952145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592426.958414] jbd2/md10-8 D ffff9c6a026830c0 0 66563 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592426.965682] Call Trace: [Tue Mar 17 17:59:02 2020][8592426.968338] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592426.973490] [] jbd2_journal_commit_transaction+0x23c/0x19b0 [jbd2] [Tue Mar 17 17:59:02 2020][8592426.981510] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:59:02 2020][8592426.987432] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:02 2020][8592426.993112] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592426.999122] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:02 2020][8592427.004781] [] ? try_to_del_timer_sync+0x5e/0x90 [Tue Mar 17 17:59:02 2020][8592427.011247] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:02 2020][8592427.017262] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592427.023278] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:02 2020][8592427.029737] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592427.034794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.041074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592427.047690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.053980] kmmpd-md4 D ffff9c8a2d8b9040 0 66626 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592427.061250] Call Trace: [Tue Mar 17 17:59:02 2020][8592427.063893] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592427.069209] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592427.074363] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:02 2020][8592427.080387] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:02 2020][8592427.086570] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:02 2020][8592427.092425] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592427.097739] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:02 2020][8592427.103919] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:02 2020][8592427.109334] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:02 2020][8592427.114735] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:02 2020][8592427.120308] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592427.125635] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:02 2020][8592427.132077] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.138172] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:02 2020][8592427.144046] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592427.150840] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592427.156784] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:02 2020][8592427.163499] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592427.168558] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 17:59:02 2020][8592427.174844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.181114] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592427.187736] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.194026] jbd2/md4-8 D ffff9c8a2d8bc100 0 66640 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592427.201313] Call Trace: [Tue Mar 17 17:59:02 2020][8592427.203944] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592427.209094] [] jbd2_journal_commit_transaction+0x23c/0x19b0 [jbd2] [Tue Mar 17 17:59:02 2020][8592427.217113] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:02 2020][8592427.223383] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:02 2020][8592427.229066] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592427.235073] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:02 2020][8592427.240732] [] ? try_to_del_timer_sync+0x5e/0x90 [Tue Mar 17 17:59:02 2020][8592427.247189] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:02 2020][8592427.253197] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:02 2020][8592427.259213] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:02 2020][8592427.265686] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592427.270747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.277036] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592427.283651] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.289923] ldiskfs-rsv-con S ffff9c5ae9458000 0 66697 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592427.297223] Call Trace: [Tue Mar 17 17:59:02 2020][8592427.299857] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592427.305005] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:02 2020][8592427.310860] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:02 2020][8592427.317056] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592427.322115] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.328403] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592427.335020] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.341304] ldiskfs-rsv-con S ffff9c6a017f2080 0 66709 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592427.348583] Call Trace: [Tue Mar 17 17:59:02 2020][8592427.351218] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592427.356366] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:02 2020][8592427.362219] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:02 2020][8592427.368423] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:02 2020][8592427.373481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.379788] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:02 2020][8592427.386404] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:02 2020][8592427.392678] kmmpd-md8 D ffff9c6a017f0000 0 66745 2 0x00000080 [Tue Mar 17 17:59:02 2020][8592427.399972] Call Trace: [Tue Mar 17 17:59:02 2020][8592427.402601] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:02 2020][8592427.407914] [] schedule+0x29/0x70 [Tue Mar 17 17:59:02 2020][8592427.413075] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:02 2020][8592427.419087] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:03 2020][8592427.425274] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:03 2020][8592427.431141] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.436461] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:03 2020][8592427.442642] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:03 2020][8592427.448064] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:03 2020][8592427.453468] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:03 2020][8592427.459048] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.464383] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:03 2020][8592427.470825] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.476919] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:03 2020][8592427.482784] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.489593] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.495523] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.502241] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592427.507296] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.513570] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592427.520197] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.526467] kmmpd-md0 D ffff9c6a017f5140 0 66759 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592427.533742] Call Trace: [Tue Mar 17 17:59:03 2020][8592427.536384] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.541700] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592427.546869] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:03 2020][8592427.552884] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:03 2020][8592427.559070] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:03 2020][8592427.564918] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.570234] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:03 2020][8592427.576421] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:03 2020][8592427.581845] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:03 2020][8592427.587244] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:03 2020][8592427.592818] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.598145] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:03 2020][8592427.604603] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.610696] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:03 2020][8592427.616550] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.623349] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.629276] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:03 2020][8592427.635992] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592427.641049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.647314] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592427.653942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.660210] jbd2/md8-8 D ffff9c6a017f4100 0 66764 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592427.667480] Call Trace: [Tue Mar 17 17:59:03 2020][8592427.670135] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592427.675286] [] jbd2_journal_commit_transaction+0x23c/0x19b0 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.683305] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:03 2020][8592427.689575] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:03 2020][8592427.695255] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:03 2020][8592427.701266] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592427.706932] [] ? try_to_del_timer_sync+0x5e/0x90 [Tue Mar 17 17:59:03 2020][8592427.713392] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.719405] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:03 2020][8592427.725421] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.731882] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592427.736937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.743228] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592427.749843] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.756115] jbd2/md0-8 D ffff9c8a04e96180 0 66788 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592427.763403] Call Trace: [Tue Mar 17 17:59:03 2020][8592427.766031] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.771343] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592427.776497] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:03 2020][8592427.782524] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 17:59:03 2020][8592427.789658] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:03 2020][8592427.795513] [] ? delayacct_end+0x8f/0xb0 [Tue Mar 17 17:59:03 2020][8592427.801262] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.806583] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:03 2020][8592427.812785] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:03 2020][8592427.818186] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:03 2020][8592427.823587] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:03 2020][8592427.829174] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:03 2020][8592427.834490] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:03 2020][8592427.840940] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.847067] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:03 2020][8592427.852911] [] jbd2_journal_commit_transaction+0x1781/0x19b0 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.861021] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.867034] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:03 2020][8592427.873056] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:03 2020][8592427.879496] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592427.884550] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.890837] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592427.897455] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.903727] ldlm_cb00_000 S ffff9c8a04e95140 0 66873 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592427.911024] Call Trace: [Tue Mar 17 17:59:03 2020][8592427.913654] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592427.919758] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592427.924959] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592427.931834] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592427.937614] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592427.944070] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592427.949763] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592427.957331] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592427.962411] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.968686] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592427.975314] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592427.981582] lc_watchdogd S ffff9c6a02682080 0 66874 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592427.988849] Call Trace: [Tue Mar 17 17:59:03 2020][8592427.991494] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592427.996645] [] lcw_dispatch_main+0x749/0x940 [libcfs] [Tue Mar 17 17:59:03 2020][8592428.003540] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:03 2020][8592428.009557] [] ? lc_watchdog_add+0x790/0x790 [libcfs] [Tue Mar 17 17:59:03 2020][8592428.016433] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.021520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.027791] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592428.034420] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.040690] ldlm_cb00_001 S ffff9c6a02680000 0 66875 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592428.047965] Call Trace: [Tue Mar 17 17:59:03 2020][8592428.050595] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592428.056702] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592428.061875] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.068750] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592428.074542] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.081005] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592428.086696] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.094281] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.099336] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.105605] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592428.112238] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.118507] ldlm_cb01_000 S ffff9c7a2e525140 0 66876 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592428.125794] Call Trace: [Tue Mar 17 17:59:03 2020][8592428.128423] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592428.134524] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592428.139730] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.146605] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592428.152380] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.158854] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592428.164547] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.172120] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.177196] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.183465] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592428.190085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.196367] ldlm_cb01_001 S ffff9c6a223a6180 0 66878 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592428.203656] Call Trace: [Tue Mar 17 17:59:03 2020][8592428.206285] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592428.212385] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592428.217568] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.224441] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592428.230217] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.236677] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592428.242364] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.249938] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.255004] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.261301] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592428.267923] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.274211] ldlm_cb02_000 S ffff9c6a223a4100 0 66879 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592428.281482] Call Trace: [Tue Mar 17 17:59:03 2020][8592428.284112] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592428.290213] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592428.295405] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.302289] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:03 2020][8592428.308905] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592428.314697] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.321158] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592428.326850] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.334436] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.339492] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.345766] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:03 2020][8592428.352395] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.358683] ldlm_cb02_001 S ffff9c6a223a2080 0 66880 2 0x00000080 [Tue Mar 17 17:59:03 2020][8592428.365957] Call Trace: [Tue Mar 17 17:59:03 2020][8592428.368585] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:03 2020][8592428.374691] [] schedule+0x29/0x70 [Tue Mar 17 17:59:03 2020][8592428.379876] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.386752] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:03 2020][8592428.392549] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.398990] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:03 2020][8592428.404685] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:03 2020][8592428.412271] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:03 2020][8592428.417329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:03 2020][8592428.423602] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.430230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.436500] ldlm_cb03_000 S ffff9c6a223a0000 0 66881 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.443776] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.446433] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.452533] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.457722] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.464595] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.470372] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.476831] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:04 2020][8592428.482521] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.490095] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.495170] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.501453] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.508068] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.514348] ldlm_cb03_001 S ffff9c6a223a1040 0 66882 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.521620] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.524249] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.530351] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.535541] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.542415] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.548197] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.554659] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:04 2020][8592428.560362] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.567931] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.573005] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.579274] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.585886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.592166] ldlm_cn00_000 S ffff9c6a0538e180 0 66883 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.599438] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.602069] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.608168] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.613354] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.620247] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:04 2020][8592428.626864] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.632662] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.639139] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.646726] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.651779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.658061] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.664676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.670941] ldlm_cn00_001 S ffff9c6a0538c100 0 66884 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.678228] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.680856] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.686948] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.692146] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.699023] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:04 2020][8592428.705643] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.711434] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.717912] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.725480] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.730548] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.736816] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.743432] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.749720] ldlm_cn00_002 S ffff9c6a0538a080 0 66885 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.756992] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.759620] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.765722] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.770914] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.777794] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.783568] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.790056] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.797624] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.802684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.808981] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.815598] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.821864] ldlm_cn01_000 S ffff9c6a0538d140 0 66886 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.829150] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.831778] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.837874] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.843061] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.849936] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:04 2020][8592428.856571] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.862363] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.868833] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.876418] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.881473] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.887755] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.894377] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.900657] ldlm_cn01_001 S ffff9c6a2373e180 0 66887 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.907931] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.910561] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.916675] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.921862] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.928743] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592428.934541] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.941017] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592428.948590] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592428.953666] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.959935] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592428.966556] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592428.972845] ldlm_cn01_002 S ffff9c6a23739040 0 66888 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592428.980116] Call Trace: [Tue Mar 17 17:59:04 2020][8592428.982744] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592428.988851] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592428.994024] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.000903] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.006699] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.013169] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.020754] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.025817] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.032099] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.038714] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.045008] ldlm_cn02_000 S ffff9c6a2373d140 0 66889 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.052283] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.054913] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.061014] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592429.066205] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.073077] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.078852] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.085339] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.092909] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.097983] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.104266] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.110882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.117162] ldlm_cn02_001 S ffff9c6a2373c100 0 66890 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.124434] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.127075] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.133174] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592429.138364] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.145239] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.151036] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.157526] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.165093] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.170182] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.176470] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.183085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.189373] ldlm_cn02_002 S ffff9c6a2fe71040 0 66891 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.196646] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.199275] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.205376] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592429.210567] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.217448] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.223223] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.229713] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.237286] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.242339] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.248620] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.255236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.261510] ldlm_cn03_000 S ffff9c6a2fe70000 0 66892 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.268797] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.271427] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.277526] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592429.282733] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.289607] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.295381] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.301870] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.309438] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.314497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.320779] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.327394] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.333659] ldlm_cn03_001 S ffff9c6a32878000 0 66893 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.340946] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.343576] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.349676] [] schedule+0x29/0x70 [Tue Mar 17 17:59:04 2020][8592429.354861] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.361767] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:04 2020][8592429.367541] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.374029] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:04 2020][8592429.381595] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:04 2020][8592429.386649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.392929] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:04 2020][8592429.399545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:04 2020][8592429.405818] ldlm_cn03_002 S ffff9c6a32879040 0 66894 2 0x00000080 [Tue Mar 17 17:59:04 2020][8592429.413104] Call Trace: [Tue Mar 17 17:59:04 2020][8592429.415734] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:04 2020][8592429.421826] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.427009] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.433884] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.439666] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.446155] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.453731] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.458790] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.465085] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.471704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.477984] ldlm_bl_01 S ffff9c6a3287e180 0 66895 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.485255] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.487887] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 17:59:05 2020][8592429.493750] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 17:59:05 2020][8592429.501159] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.506330] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.513384] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.519178] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.526748] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.531837] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.538111] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.544729] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.551011] ldlm_bl_02 S ffff9c6a3287d140 0 66896 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.558283] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.560910] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 17:59:05 2020][8592429.566762] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 17:59:05 2020][8592429.574176] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.579347] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.586399] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.592202] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.599774] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.604832] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.611115] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.617730] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.624003] ldlm_elt S ffff9c6a3287c100 0 66897 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.631289] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.633922] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592429.640021] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.645224] [] expired_lock_main+0x1c2/0xc00 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.652103] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.657884] [] ? ldlm_server_glimpse_ast+0x3f0/0x3f0 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.665472] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.670530] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.676816] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.683433] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.689719] ll_cfg_requeue S ffff9c6a3287a080 0 66899 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.696994] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.699621] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592429.705715] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.710893] [] mgc_requeue_thread+0x5f5/0x870 [mgc] [Tue Mar 17 17:59:05 2020][8592429.717596] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.723346] [] ? mgc_process_config+0x13f0/0x13f0 [mgc] [Tue Mar 17 17:59:05 2020][8592429.730409] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.735469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.741758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.748379] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.754662] ll_ost00_000 S ffff9c69c1680000 0 66904 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.761933] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.764561] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592429.770668] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.775858] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.782735] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.788524] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.794994] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.802567] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.807639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.813909] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.820537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.826821] ll_ost00_001 S ffff9c69c1681040 0 66906 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.834091] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.836722] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592429.842836] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.848008] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.854887] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.860682] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.867153] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.874725] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.879800] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.886084] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.892697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.898979] ll_ost00_002 S ffff9c69c1682080 0 66907 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.906250] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.908880] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592429.914982] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.920173] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.927053] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592429.932829] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.939317] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592429.946892] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592429.951980] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.958255] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592429.964875] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592429.971154] ll_ost01_000 D ffff9c69c16830c0 0 66908 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592429.978428] Call Trace: [Tue Mar 17 17:59:05 2020][8592429.981056] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:05 2020][8592429.986368] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592429.991521] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:05 2020][8592429.997540] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:05 2020][8592430.004771] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:05 2020][8592430.010536] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:05 2020][8592430.017068] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:05 2020][8592430.022909] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:05 2020][8592430.028245] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:05 2020][8592430.034427] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:05 2020][8592430.039833] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:05 2020][8592430.045249] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:05 2020][8592430.050822] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:05 2020][8592430.056136] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:05 2020][8592430.062604] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.068711] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 17:59:05 2020][8592430.075587] [] ? __getblk+0x2d/0x300 [Tue Mar 17 17:59:05 2020][8592430.081010] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:05 2020][8592430.088595] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.096866] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.104541] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:05 2020][8592430.111513] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.118670] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.126161] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.133131] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:05 2020][8592430.139953] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 17:59:05 2020][8592430.146057] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:05 2020][8592430.152793] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.160589] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.168600] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:05 2020][8592430.175097] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.182183] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.189944] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:05 2020][8592430.197199] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.205059] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.212053] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592430.217832] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.224323] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.231898] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592430.236960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.243249] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592430.249864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.256139] ll_ost01_001 S ffff9c69c1684100 0 66909 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592430.263432] Call Trace: [Tue Mar 17 17:59:05 2020][8592430.266063] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592430.272162] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592430.277346] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.284229] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592430.290011] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.296500] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.304074] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592430.309136] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.315424] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592430.322041] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.328342] ll_ost01_002 S ffff9c69c1685140 0 66910 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592430.335616] Call Trace: [Tue Mar 17 17:59:05 2020][8592430.338261] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:05 2020][8592430.344357] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592430.349539] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.356412] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:05 2020][8592430.362187] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.368677] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:05 2020][8592430.376250] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:05 2020][8592430.381303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.387598] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:05 2020][8592430.394216] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:05 2020][8592430.400483] ll_ost02_000 D ffff9c69c1686180 0 66911 2 0x00000080 [Tue Mar 17 17:59:05 2020][8592430.407768] Call Trace: [Tue Mar 17 17:59:05 2020][8592430.410397] [] schedule+0x29/0x70 [Tue Mar 17 17:59:05 2020][8592430.415541] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:05 2020][8592430.422606] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:05 2020][8592430.428619] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.435860] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.442571] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.450089] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:06 2020][8592430.456274] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.463071] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.470401] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.477978] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.485131] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:06 2020][8592430.491408] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:06 2020][8592430.497510] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:06 2020][8592430.504256] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.512022] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.520025] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:06 2020][8592430.526532] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.533618] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.541364] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:06 2020][8592430.548632] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.556495] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.563479] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:06 2020][8592430.568914] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.575389] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.582976] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592430.588031] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592430.594333] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592430.600953] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592430.607219] ll_ost02_001 D ffff9c69c16c0000 0 66919 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592430.614503] Call Trace: [Tue Mar 17 17:59:06 2020][8592430.617134] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592430.622284] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.629350] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:06 2020][8592430.635363] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.642603] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.649312] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.656806] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:06 2020][8592430.662991] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:06 2020][8592430.669788] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.677120] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.684696] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:06 2020][8592430.691835] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:06 2020][8592430.698124] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:06 2020][8592430.704219] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:06 2020][8592430.710951] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.718752] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.726763] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:06 2020][8592430.733255] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.740336] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.748085] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:06 2020][8592430.755349] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.763203] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.770171] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:06 2020][8592430.775623] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.782098] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.789670] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592430.794751] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592430.801024] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592430.807637] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592430.813924] ll_ost02_002 S ffff9c69c16c1040 0 66920 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592430.821198] Call Trace: [Tue Mar 17 17:59:06 2020][8592430.823847] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 17:59:06 2020][8592430.831159] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592430.836320] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:06 2020][8592430.842332] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:06 2020][8592430.848893] [] ldlm_completion_ast+0x430/0x860 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.856000] [] ? ldlm_completion_ast_async+0x300/0x300 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.863771] [] ? ldlm_lock_enqueue+0x3ff/0xa20 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.870824] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592430.876612] [] ldlm_cli_enqueue_local+0x231/0x830 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.883946] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.891989] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.899733] [] ofd_destroy_by_fid+0x1dd/0x4a0 [ofd] [Tue Mar 17 17:59:06 2020][8592430.906463] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.914259] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.922267] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:06 2020][8592430.928759] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.935841] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.943587] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:06 2020][8592430.950854] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.958707] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.965676] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:06 2020][8592430.971128] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.977605] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592430.985175] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592430.990265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592430.996536] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.003148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.009430] ll_ost03_000 S ffff9c69c16c2080 0 66921 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.016699] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.019329] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:06 2020][8592431.025423] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.030604] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.037480] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.043263] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.049765] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.057340] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.062409] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.068679] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.075313] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.081583] ll_ost03_001 S ffff9c69c16c30c0 0 66922 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.088853] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.091494] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.096667] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.103561] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.109350] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.115821] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.123406] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.128460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.134743] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.141358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.147633] ll_ost03_002 S ffff9c69c16c4100 0 66923 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.154919] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.157545] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:06 2020][8592431.163655] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.168854] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.175731] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.181527] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.187995] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.195568] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.200645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.206914] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.213533] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.219814] ll_ost_create00 S ffff9c69c16c5140 0 66924 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.227086] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.229714] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:06 2020][8592431.235809] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.241001] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.247882] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.253681] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.260126] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:06 2020][8592431.265828] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.273396] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.278456] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.284739] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.291367] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.297636] ll_ost_create00 S ffff9c69c16c6180 0 66925 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.304922] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.307551] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:06 2020][8592431.313653] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.318843] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.325715] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.331492] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.337951] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:06 2020][8592431.343641] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.351213] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.356297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.362568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:06 2020][8592431.369191] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:06 2020][8592431.375463] ll_ost_create01 S ffff9c69c1128000 0 66926 2 0x00000080 [Tue Mar 17 17:59:06 2020][8592431.382746] Call Trace: [Tue Mar 17 17:59:06 2020][8592431.385378] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:06 2020][8592431.391469] [] schedule+0x29/0x70 [Tue Mar 17 17:59:06 2020][8592431.396657] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.403536] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:06 2020][8592431.409320] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.415810] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:06 2020][8592431.423383] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:06 2020][8592431.428458] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.434727] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.441348] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.447637] ll_ost_create01 S ffff9c69c1129040 0 66930 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.454909] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.457537] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:07 2020][8592431.463631] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.468833] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.475712] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:07 2020][8592431.481496] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.487982] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.495550] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.500628] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.506895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.513514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.519797] ll_ost_create02 S ffff9c69c112a080 0 66936 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.527068] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.529696] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:07 2020][8592431.535787] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.540987] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.547864] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:07 2020][8592431.553643] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.560134] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.567701] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.572761] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.579050] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.585666] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.591930] ll_ost_create02 S ffff9c69c112b0c0 0 66938 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.599219] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.601846] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:07 2020][8592431.607938] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.613120] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.619996] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:07 2020][8592431.625781] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.632277] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.639849] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.644902] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.651200] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.657818] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.664090] ll_ost_create03 S ffff9c69c112c100 0 66939 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.671377] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.674005] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:07 2020][8592431.680097] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.685281] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.692154] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:07 2020][8592431.697929] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.704418] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.712006] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.717063] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.723351] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.729968] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.736240] ll_ost_create03 S ffff9c69c112d140 0 66940 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.743527] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.746155] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:07 2020][8592431.752250] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.757441] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.764316] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:07 2020][8592431.770099] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.776598] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.784170] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.789245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.795513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592431.802126] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592431.808407] ll_ost_io00_000 D ffff9c69c112e180 0 66946 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592431.815677] Call Trace: [Tue Mar 17 17:59:07 2020][8592431.818306] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592431.823457] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:07 2020][8592431.830539] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:07 2020][8592431.836554] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:07 2020][8592431.843794] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:07 2020][8592431.849632] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:07 2020][8592431.856363] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592431.863681] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:07 2020][8592431.869868] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:07 2020][8592431.876679] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592431.884003] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:07 2020][8592431.891594] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592431.898756] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:07 2020][8592431.905551] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:07 2020][8592431.911786] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.918523] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.925186] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.932240] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:07 2020][8592431.937786] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.944871] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.952634] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:07 2020][8592431.959891] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.967780] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.974748] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:07 2020][8592431.980193] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.986670] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592431.994255] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592431.999312] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.005602] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592432.012216] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.018481] ll_ost_io00_001 D ffff9c69c09b0000 0 66950 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592432.025773] Call Trace: [Tue Mar 17 17:59:07 2020][8592432.028406] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592432.033556] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:07 2020][8592432.040638] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:07 2020][8592432.046651] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:07 2020][8592432.053892] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:07 2020][8592432.059731] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:07 2020][8592432.066456] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.073771] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:07 2020][8592432.079973] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:07 2020][8592432.086768] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.094090] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.101677] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.108821] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:07 2020][8592432.115630] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:07 2020][8592432.121850] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.128588] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.135249] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.142304] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:07 2020][8592432.147837] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.154923] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.162693] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:07 2020][8592432.169974] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.177832] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.184813] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:07 2020][8592432.190243] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.196716] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.204303] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592432.209358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.215640] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592432.222256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.228529] ll_ost_io00_002 D ffff9c69c09b1040 0 66951 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592432.235824] Call Trace: [Tue Mar 17 17:59:07 2020][8592432.238454] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592432.243605] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.250678] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:07 2020][8592432.257814] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:07 2020][8592432.263824] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:07 2020][8592432.270025] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:07 2020][8592432.276817] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:07 2020][8592432.283035] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.289801] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.296445] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.303537] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:07 2020][8592432.310846] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:07 2020][8592432.316617] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:07 2020][8592432.323171] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.330604] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.337696] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.345448] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:07 2020][8592432.352715] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.360613] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.367579] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:07 2020][8592432.373031] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.379507] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:07 2020][8592432.387094] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:07 2020][8592432.392148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.398430] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:07 2020][8592432.405044] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:07 2020][8592432.411312] ll_ost_io01_000 D ffff9c69c09b2080 0 66952 2 0x00000080 [Tue Mar 17 17:59:07 2020][8592432.418606] Call Trace: [Tue Mar 17 17:59:07 2020][8592432.421234] [] schedule+0x29/0x70 [Tue Mar 17 17:59:07 2020][8592432.426379] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.433456] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:08 2020][8592432.439472] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:08 2020][8592432.445661] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:08 2020][8592432.452469] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:08 2020][8592432.458691] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.465430] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.472091] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.479163] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:08 2020][8592432.486506] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:08 2020][8592432.492259] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:08 2020][8592432.497787] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.504886] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.512636] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:08 2020][8592432.519892] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.527766] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.534731] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:08 2020][8592432.540176] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.546653] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.554252] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:08 2020][8592432.559310] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592432.565593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:08 2020][8592432.572209] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592432.578496] ll_ost_io01_001 D ffff9c69c09b30c0 0 66953 2 0x00000080 [Tue Mar 17 17:59:08 2020][8592432.585773] Call Trace: [Tue Mar 17 17:59:08 2020][8592432.588405] [] schedule+0x29/0x70 [Tue Mar 17 17:59:08 2020][8592432.593549] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.600614] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:08 2020][8592432.606627] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.613879] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:08 2020][8592432.619721] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.626450] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.633758] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:08 2020][8592432.639955] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.646749] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.654065] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.661652] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.668796] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:08 2020][8592432.675588] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:08 2020][8592432.681836] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.688572] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.695219] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.702302] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:08 2020][8592432.709611] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:08 2020][8592432.715365] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:08 2020][8592432.720907] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.727990] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.735752] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:08 2020][8592432.743007] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.750868] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.757860] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:08 2020][8592432.763296] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.769786] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.777357] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:08 2020][8592432.782424] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592432.788694] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:08 2020][8592432.795321] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592432.801592] ll_ost_io01_002 D ffff9c69c09b4100 0 66954 2 0x00000080 [Tue Mar 17 17:59:08 2020][8592432.808869] Call Trace: [Tue Mar 17 17:59:08 2020][8592432.811510] [] schedule+0x29/0x70 [Tue Mar 17 17:59:08 2020][8592432.816657] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.823723] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:08 2020][8592432.829733] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.836976] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.843682] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.851438] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:08 2020][8592432.857624] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592432.864420] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.871751] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.879327] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592432.886496] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:08 2020][8592432.892771] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:08 2020][8592432.899570] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:08 2020][8592432.905807] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.912507] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 17:59:08 2020][8592432.918473] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.925120] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.932176] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:08 2020][8592432.937714] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.944799] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.952579] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:08 2020][8592432.959831] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.967692] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.974671] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:08 2020][8592432.980101] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.986575] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592432.994163] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:08 2020][8592432.999218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.005505] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:08 2020][8592433.012123] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.018424] ll_ost_io02_000 D ffff9c69c09b5140 0 66955 2 0x00000080 [Tue Mar 17 17:59:08 2020][8592433.025699] Call Trace: [Tue Mar 17 17:59:08 2020][8592433.028330] [] schedule+0x29/0x70 [Tue Mar 17 17:59:08 2020][8592433.033482] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:08 2020][8592433.040547] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:08 2020][8592433.046558] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592433.053798] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:08 2020][8592433.059635] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:08 2020][8592433.066366] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.073687] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:08 2020][8592433.079874] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:08 2020][8592433.086684] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.094006] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.101592] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.108735] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:08 2020][8592433.115530] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:08 2020][8592433.121759] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.128496] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.135157] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.142210] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:08 2020][8592433.147758] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.154846] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.162619] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:08 2020][8592433.169870] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.177745] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.184713] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:08 2020][8592433.190150] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.196639] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.204211] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:08 2020][8592433.209266] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.215568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:08 2020][8592433.222187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.228461] ll_ost_io02_001 D ffff9c69c09b6180 0 66967 2 0x00000080 [Tue Mar 17 17:59:08 2020][8592433.235747] Call Trace: [Tue Mar 17 17:59:08 2020][8592433.238378] [] schedule+0x29/0x70 [Tue Mar 17 17:59:08 2020][8592433.243529] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.250593] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:08 2020][8592433.256606] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:08 2020][8592433.262794] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:08 2020][8592433.269616] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:08 2020][8592433.275832] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.282563] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.289224] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.296269] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:08 2020][8592433.301811] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.308894] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.316642] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:08 2020][8592433.323907] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.331763] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.338755] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:08 2020][8592433.344192] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.350666] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:08 2020][8592433.358253] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:08 2020][8592433.363329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.369615] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:08 2020][8592433.376230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:08 2020][8592433.382519] ll_ost_io02_002 D ffff9c69c0b08000 0 66968 2 0x00000080 [Tue Mar 17 17:59:08 2020][8592433.389791] Call Trace: [Tue Mar 17 17:59:08 2020][8592433.392423] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:08 2020][8592433.397741] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:08 2020][8592433.404392] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:08 2020][8592433.410748] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:08 2020][8592433.416064] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:08 2020][8592433.421578] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:08 2020][8592433.428548] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.435945] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:09 2020][8592433.442238] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.450593] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 17:59:09 2020][8592433.456960] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.464224] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.471885] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:09 2020][8592433.477488] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.485153] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.493074] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.500487] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.507547] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.515746] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.522713] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.529942] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.537281] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:09 2020][8592433.544073] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:09 2020][8592433.550291] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.557007] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 17:59:09 2020][8592433.562669] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:09 2020][8592433.568438] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:09 2020][8592433.574993] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.582426] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.589520] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.597271] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:09 2020][8592433.604525] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.612413] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.619374] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:09 2020][8592433.624807] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.631289] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.638857] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:09 2020][8592433.643927] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592433.650194] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:09 2020][8592433.656816] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592433.663103] ll_ost_io03_000 D ffff9c69c0b09040 0 66970 2 0x00000080 [Tue Mar 17 17:59:09 2020][8592433.670393] Call Trace: [Tue Mar 17 17:59:09 2020][8592433.673022] [] schedule+0x29/0x70 [Tue Mar 17 17:59:09 2020][8592433.678173] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.685238] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:09 2020][8592433.691249] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.698492] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.705197] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.712941] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:09 2020][8592433.719146] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.725946] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.733262] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.740863] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.748002] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:09 2020][8592433.754277] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:09 2020][8592433.761085] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:09 2020][8592433.767306] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.774015] [] ? ___slab_alloc+0x24e/0x4f0 [Tue Mar 17 17:59:09 2020][8592433.779987] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.786636] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.793688] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:09 2020][8592433.799757] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.807189] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.814266] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.822030] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:09 2020][8592433.829286] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.837148] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.844127] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:09 2020][8592433.849558] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.856044] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.863611] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:09 2020][8592433.868691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592433.874966] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:09 2020][8592433.881584] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592433.887867] ll_ost_io03_001 D ffff9c69c0b0a080 0 66973 2 0x00000080 [Tue Mar 17 17:59:09 2020][8592433.895138] Call Trace: [Tue Mar 17 17:59:09 2020][8592433.897767] [] schedule+0x29/0x70 [Tue Mar 17 17:59:09 2020][8592433.902910] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.909975] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:09 2020][8592433.915986] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.923228] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:09 2020][8592433.929065] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.935812] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.943121] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:09 2020][8592433.949325] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592433.956126] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.963442] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.971029] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592433.978174] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:09 2020][8592433.984967] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:09 2020][8592433.991199] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592433.997902] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:09 2020][8592434.003962] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.011394] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.018490] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.026238] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:09 2020][8592434.033491] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.041365] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.048325] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:09 2020][8592434.053769] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.060243] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.067843] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:09 2020][8592434.072905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592434.079194] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:09 2020][8592434.085808] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592434.092074] ll_ost_io03_002 D ffff9c69c0b0b0c0 0 66974 2 0x00000080 [Tue Mar 17 17:59:09 2020][8592434.099361] Call Trace: [Tue Mar 17 17:59:09 2020][8592434.101988] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:09 2020][8592434.108604] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:09 2020][8592434.114978] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:09 2020][8592434.120293] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:09 2020][8592434.125816] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.132789] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.140184] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:09 2020][8592434.146481] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.154831] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 17:59:09 2020][8592434.161199] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.168456] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.176116] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:09 2020][8592434.181696] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.189373] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.197311] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.204714] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.211788] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.219966] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:09 2020][8592434.226950] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.234180] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:09 2020][8592434.241508] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:09 2020][8592434.248304] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:09 2020][8592434.254522] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.261264] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.268693] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.275801] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.283551] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:09 2020][8592434.290822] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.298684] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.305665] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:09 2020][8592434.311094] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.317568] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.325154] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:09 2020][8592434.330211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592434.336513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:09 2020][8592434.343132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592434.349398] ll_ost_seq00_00 S ffff9c69c0b0c100 0 66976 2 0x00000080 [Tue Mar 17 17:59:09 2020][8592434.356684] Call Trace: [Tue Mar 17 17:59:09 2020][8592434.359314] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:09 2020][8592434.365436] [] schedule+0x29/0x70 [Tue Mar 17 17:59:09 2020][8592434.370623] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.377497] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:09 2020][8592434.383296] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.389743] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:09 2020][8592434.395439] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:09 2020][8592434.403038] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:09 2020][8592434.408097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:09 2020][8592434.414380] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.420995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.427269] ll_ost_seq00_00 S ffff9c69c0b0d140 0 66977 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.434554] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.437182] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.443275] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.448460] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.455357] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.461133] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.467594] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.473286] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.480870] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.485925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.492192] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.498827] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.505095] ll_ost_seq01_00 S ffff9c69c0b0e180 0 66978 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.512364] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.515008] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.521104] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.526288] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.533168] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.538950] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.545411] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.551103] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.558674] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.563748] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.570020] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.576641] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.582929] ll_ost_seq01_00 S ffff9c69c05a8000 0 66980 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.590200] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.592829] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.598923] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.604113] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.610985] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.616762] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.623222] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.628914] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.636484] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.641560] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.647829] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.654450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.660729] ll_ost_seq02_00 S ffff9c69c05a9040 0 66981 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.668002] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.670632] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.676731] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.681915] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.688797] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.694580] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.701040] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.706732] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.714318] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.719371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.725652] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.732269] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.738534] ll_ost_seq02_00 S ffff9c69c05aa080 0 66984 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.745821] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.748451] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.754552] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.759743] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.766623] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.772399] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.778857] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.784549] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.792120] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.797187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.803456] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.810071] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.816372] ll_ost_seq03_00 S ffff9c69c05ac100 0 66992 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.823648] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.826290] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.832387] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.837578] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.844460] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.850243] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.856706] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.862400] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.869989] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.875059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.881326] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.887955] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.894222] ll_ost_seq03_00 S ffff9c69c05ad140 0 66995 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.901492] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.904136] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.910231] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.915416] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.922296] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592434.928079] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.934553] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592434.940246] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592434.947820] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592434.952893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.959165] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592434.965785] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592434.972073] ll_ost_out00_00 S ffff9c69bc738000 0 67041 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592434.979344] Call Trace: [Tue Mar 17 17:59:10 2020][8592434.981975] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592434.988075] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592434.993275] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.000149] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.005933] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.012393] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.018085] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.025654] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592435.030721] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.036991] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592435.043614] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.049901] ll_ost_out00_00 S ffff9c69bc739040 0 67044 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592435.057189] Call Trace: [Tue Mar 17 17:59:10 2020][8592435.059820] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592435.065920] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592435.071103] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.077975] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.083750] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.090209] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.095903] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.103488] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592435.108545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.114817] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592435.121458] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.127731] ll_ost_out01_00 S ffff9c69bc73a080 0 67045 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592435.134999] Call Trace: [Tue Mar 17 17:59:10 2020][8592435.137628] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592435.143736] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592435.148909] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.155785] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.161577] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.168023] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.173719] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.181305] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592435.186363] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.192653] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592435.199265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.205531] ll_ost_out01_00 S ffff9c69bc73b0c0 0 67046 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592435.212819] Call Trace: [Tue Mar 17 17:59:10 2020][8592435.215447] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592435.221540] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592435.226731] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.233620] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.239412] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.245874] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.251564] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.259152] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592435.264206] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.270481] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592435.277108] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.283377] ll_ost_out02_00 S ffff9c69bc73c100 0 67047 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592435.290654] Call Trace: [Tue Mar 17 17:59:10 2020][8592435.293298] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592435.299393] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592435.304590] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.311466] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.317240] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.323700] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.329390] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.336977] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:10 2020][8592435.342034] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.348310] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:10 2020][8592435.354956] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:10 2020][8592435.361230] ll_ost_out02_00 S ffff9c69bc73d140 0 67048 2 0x00000080 [Tue Mar 17 17:59:10 2020][8592435.368500] Call Trace: [Tue Mar 17 17:59:10 2020][8592435.371141] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:10 2020][8592435.377238] [] schedule+0x29/0x70 [Tue Mar 17 17:59:10 2020][8592435.382429] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.389304] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:10 2020][8592435.395085] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.401546] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:10 2020][8592435.407237] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:10 2020][8592435.414836] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.419895] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.426176] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.432792] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.439080] ll_ost_out03_00 S ffff9c69bc73e180 0 67049 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.446351] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.448980] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592435.455075] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.460264] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.467138] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592435.472921] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.479421] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.486994] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.492068] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.498337] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.504949] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.511231] ll_ost_out03_00 S ffff9c69bc798000 0 67050 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.518501] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.521130] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592435.527223] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.532420] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.539297] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592435.545081] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.551539] [] ? __schedule+0x42a/0x860 [Tue Mar 17 17:59:11 2020][8592435.557232] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.564803] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.569870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.576140] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.582760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.589043] ll_evictor S ffff9c69bc799040 0 67051 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.596312] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.598969] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592435.605068] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.610253] [] ping_evictor_main+0x4be/0x700 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.617134] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592435.622916] [] ? ptlrpc_del_timeout_client+0x190/0x190 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.630676] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.635733] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.642021] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.648635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.654900] ldiskfs-rsv-con S ffff9c69bc79a080 0 67058 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.662196] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.664826] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.669979] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:11 2020][8592435.675830] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:11 2020][8592435.682012] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.687088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.693356] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.699969] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.706249] ldiskfs-rsv-con S ffff9c5a1fef6180 0 67082 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.713541] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.716168] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.721315] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:11 2020][8592435.727161] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:11 2020][8592435.733344] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.738398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.744688] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.751303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.757577] ll_ost00_003 S ffff9c5997fc0000 0 67108 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.764861] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.767489] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592435.773585] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.778788] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.785664] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592435.791455] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.797925] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.805509] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.810566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.816854] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.823469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.829737] ll_ost03_003 S ffff9c5a1fef0000 0 67232 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.837030] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.839662] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592435.845760] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.850943] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.857817] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592435.863598] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.870088] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.877654] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592435.882708] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.889011] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592435.895628] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592435.901909] ll_ost02_003 D ffff9c5a1fef30c0 0 67233 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592435.909179] Call Trace: [Tue Mar 17 17:59:11 2020][8592435.911811] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:11 2020][8592435.917825] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592435.924473] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:11 2020][8592435.930835] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:11 2020][8592435.936192] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.943206] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.950303] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.958053] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:11 2020][8592435.965317] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.973171] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.980155] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:11 2020][8592435.985584] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.992058] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592435.999644] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592436.004700] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 17:59:11 2020][8592436.010981] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.017264] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592436.023878] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.030159] ll_ost01_003 S ffff9c5a144f8000 0 67543 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592436.037430] Call Trace: [Tue Mar 17 17:59:11 2020][8592436.040060] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592436.046153] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592436.051336] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592436.058216] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592436.063992] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592436.070480] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592436.078047] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592436.083135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.089408] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592436.096021] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.102302] kmmpd-md2 D ffff9c5a2ab88000 0 67554 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592436.109574] Call Trace: [Tue Mar 17 17:59:11 2020][8592436.112203] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.117523] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592436.122677] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:11 2020][8592436.128690] [] ? md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:11 2020][8592436.134879] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:11 2020][8592436.140731] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.146046] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:11 2020][8592436.152226] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:11 2020][8592436.157647] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:11 2020][8592436.163050] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:11 2020][8592436.168647] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.173976] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:11 2020][8592436.180417] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.186526] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:11 2020][8592436.192386] [] write_mmp_block+0x11d/0x160 [ldiskfs] [Tue Mar 17 17:59:11 2020][8592436.199179] [] kmmpd+0x1a8/0x430 [ldiskfs] [Tue Mar 17 17:59:11 2020][8592436.205122] [] ? __dump_mmp_msg+0x70/0x70 [ldiskfs] [Tue Mar 17 17:59:11 2020][8592436.211825] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592436.216879] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.223167] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592436.229787] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.236059] jbd2/md2-8 D ffff9c598da82080 0 67601 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592436.243354] Call Trace: [Tue Mar 17 17:59:11 2020][8592436.245981] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.251292] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592436.256461] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:11 2020][8592436.262473] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:11 2020][8592436.268309] [] ? delayacct_end+0x8f/0xb0 [Tue Mar 17 17:59:11 2020][8592436.274077] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.279392] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:11 2020][8592436.285572] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:11 2020][8592436.290992] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:11 2020][8592436.296397] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:11 2020][8592436.301975] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:11 2020][8592436.307317] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:11 2020][8592436.313765] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.319867] [] __wait_on_buffer+0x2a/0x30 [Tue Mar 17 17:59:11 2020][8592436.325730] [] jbd2_journal_commit_transaction+0xe82/0x19b0 [jbd2] [Tue Mar 17 17:59:11 2020][8592436.333737] [] kjournald2+0xc9/0x260 [jbd2] [Tue Mar 17 17:59:11 2020][8592436.339760] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:11 2020][8592436.345773] [] ? commit_timeout+0x10/0x10 [jbd2] [Tue Mar 17 17:59:11 2020][8592436.352232] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:11 2020][8592436.357290] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.363577] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:11 2020][8592436.370207] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:11 2020][8592436.376475] ll_ost02_004 S ffff9c598da830c0 0 67611 2 0x00000080 [Tue Mar 17 17:59:11 2020][8592436.383761] Call Trace: [Tue Mar 17 17:59:11 2020][8592436.386391] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:11 2020][8592436.392483] [] schedule+0x29/0x70 [Tue Mar 17 17:59:11 2020][8592436.397669] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592436.404549] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:11 2020][8592436.410331] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:11 2020][8592436.416822] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.424395] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.429483] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.435758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.442379] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.448669] ll_ost02_005 S ffff9c598da85140 0 67744 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.455939] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.458565] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592436.464673] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.469848] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.476726] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592436.482523] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.489004] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.496571] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.501639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.507908] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.514527] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.520807] ll_ost00_004 S ffff9c598da86180 0 67819 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.528082] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.530709] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592436.536802] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.541986] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.548881] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592436.554660] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.561146] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.568713] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.573784] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.580058] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.586681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.592967] ll_ost01_004 S ffff9c5981780000 0 67825 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.600239] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.602882] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592436.608979] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.614184] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.621061] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592436.626844] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.633332] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.640899] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.645959] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.652242] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.658856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.665131] ll_ost03_004 D ffff9c5981781040 0 67857 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.672424] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.675055] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.680206] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:12 2020][8592436.687272] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:12 2020][8592436.693282] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:12 2020][8592436.700526] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:12 2020][8592436.707245] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592436.714734] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:12 2020][8592436.720936] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:12 2020][8592436.727736] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592436.735081] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:12 2020][8592436.742661] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592436.749802] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:12 2020][8592436.756087] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:12 2020][8592436.762183] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:12 2020][8592436.768917] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.776703] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.784709] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:12 2020][8592436.791202] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.798284] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.806031] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:12 2020][8592436.813310] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.821167] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.828148] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:12 2020][8592436.833580] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.840053] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.847640] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.852695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.858979] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.865591] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.871857] ll_ost01_005 S ffff9c8a36a5b0c0 0 67860 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.879160] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.881790] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.886959] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.893854] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592436.899628] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.906102] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.913690] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.918744] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.925024] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592436.931657] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.937925] ll_ost01_006 S ffff9c8a36a5c100 0 67861 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592436.945218] Call Trace: [Tue Mar 17 17:59:12 2020][8592436.947848] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592436.953950] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592436.959141] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.966020] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592436.971799] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.978285] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592436.985852] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592436.990905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592436.997207] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592437.003825] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.010091] ldiskfs-rsv-con S ffff9c8a36a5e180 0 67888 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592437.017384] Call Trace: [Tue Mar 17 17:59:12 2020][8592437.020015] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592437.025156] [] rescuer_thread+0x31e/0x3c0 [Tue Mar 17 17:59:12 2020][8592437.031011] [] ? process_one_work+0x440/0x440 [Tue Mar 17 17:59:12 2020][8592437.037193] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592437.042246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.048548] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592437.055168] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.061434] ll_ost03_005 D ffff9c69bc79b0c0 0 68134 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592437.068720] Call Trace: [Tue Mar 17 17:59:12 2020][8592437.071348] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592437.076492] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:12 2020][8592437.083557] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:12 2020][8592437.089570] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:12 2020][8592437.096809] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:12 2020][8592437.103521] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592437.111038] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:12 2020][8592437.117224] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:12 2020][8592437.124036] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592437.131357] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:12 2020][8592437.138943] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:12 2020][8592437.146086] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:12 2020][8592437.152362] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:12 2020][8592437.158477] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:12 2020][8592437.165212] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.173000] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.181034] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:12 2020][8592437.187507] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.194615] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.202360] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:12 2020][8592437.209610] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.217488] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.224455] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:12 2020][8592437.229891] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.236379] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.243949] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592437.249035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.255308] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592437.261934] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.268206] ll_ost00_005 S ffff9c69bc79d140 0 68146 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592437.275482] Call Trace: [Tue Mar 17 17:59:12 2020][8592437.278124] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592437.284220] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592437.289404] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.296285] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592437.302070] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.308570] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.316141] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592437.321216] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.327485] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592437.334098] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.340387] ll_ost00_006 S ffff9c69bc79e180 0 68147 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592437.347658] Call Trace: [Tue Mar 17 17:59:12 2020][8592437.350286] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:12 2020][8592437.356411] [] schedule+0x29/0x70 [Tue Mar 17 17:59:12 2020][8592437.361598] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.368479] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:12 2020][8592437.374284] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.380754] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:12 2020][8592437.388326] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:12 2020][8592437.393393] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.399662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:12 2020][8592437.406282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:12 2020][8592437.412563] ll_ost01_007 S ffff9c69bc79c100 0 68148 2 0x00000080 [Tue Mar 17 17:59:12 2020][8592437.419837] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.422465] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592437.428564] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.433761] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.440638] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.446421] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.452911] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.460485] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.465545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.471827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.478443] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.484718] ll_ost02_006 S ffff9c8a36a59040 0 68149 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.492028] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.494658] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.499826] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.506721] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.512496] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.518970] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.526557] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.531612] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.537894] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.544510] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.550786] inconsistency_v S ffff9c8a36a5d140 0 68150 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.558086] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.560715] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592437.566807] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.571969] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:13 2020][8592437.580150] [] ? sched_clock_cpu+0x85/0xc0 [Tue Mar 17 17:59:13 2020][8592437.586077] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:13 2020][8592437.591758] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.597511] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:13 2020][8592437.604214] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.609304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.615576] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.622197] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.628478] lquota_wb_fir-O S ffff9c5a16e19040 0 68171 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.635749] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.638378] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.643518] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:13 2020][8592437.649556] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:13 2020][8592437.657386] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:13 2020][8592437.663941] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:13 2020][8592437.670751] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:13 2020][8592437.677369] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.683133] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:13 2020][8592437.689495] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.694556] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.700845] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.707460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.713726] inconsistency_v S ffff9c5a144fd140 0 68179 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.721020] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.723652] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592437.729750] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.734923] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:13 2020][8592437.743099] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:13 2020][8592437.749372] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:13 2020][8592437.755047] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.760798] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:13 2020][8592437.767505] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.772581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.778849] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.785485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.791766] lquota_wb_fir-O S ffff9c6a3735e180 0 68198 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.799039] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.801668] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.806823] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:13 2020][8592437.812842] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:13 2020][8592437.820674] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:13 2020][8592437.827226] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:13 2020][8592437.834025] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:13 2020][8592437.840643] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592437.846416] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:13 2020][8592437.852777] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.857864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.864138] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.870759] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.877040] ll_ost02_007 D ffff9c6a24f10000 0 68205 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592437.884310] Call Trace: [Tue Mar 17 17:59:13 2020][8592437.886939] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:13 2020][8592437.892946] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592437.899573] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:13 2020][8592437.905930] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:13 2020][8592437.911293] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.918301] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.925382] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.933146] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:13 2020][8592437.940402] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.948273] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.955234] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:13 2020][8592437.960687] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.967160] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:13 2020][8592437.974736] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592437.979821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.986086] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592437.992715] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592437.998984] inconsistency_v S ffff9c6a23738000 0 68207 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.006259] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.008904] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592438.015000] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.020166] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:13 2020][8592438.028347] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:13 2020][8592438.034620] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:13 2020][8592438.040308] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592438.046065] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:13 2020][8592438.052770] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592438.057844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.064114] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592438.070735] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.077015] lquota_wb_fir-O S ffff9c7a2c50e180 0 68226 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.084288] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.086916] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.092056] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:13 2020][8592438.098106] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:13 2020][8592438.105940] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:13 2020][8592438.112494] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:13 2020][8592438.119291] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:13 2020][8592438.125907] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592438.131673] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:13 2020][8592438.138032] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592438.143093] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.149375] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592438.155991] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.162265] inconsistency_v S ffff9c7a2c50a080 0 68234 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.169568] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.172211] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592438.178307] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.183488] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:13 2020][8592438.191662] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:13 2020][8592438.197938] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:13 2020][8592438.203617] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592438.209372] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:13 2020][8592438.216074] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592438.221156] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.227429] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592438.234043] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.240329] lquota_wb_fir-O S ffff9c6a362d8000 0 68253 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.247603] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.250232] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.255379] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:13 2020][8592438.261410] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:13 2020][8592438.269237] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:13 2020][8592438.275769] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:13 2020][8592438.282591] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:13 2020][8592438.289207] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592438.294966] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:13 2020][8592438.301338] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592438.306393] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.312667] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592438.319304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.325571] inconsistency_v S ffff9c7a35bb9040 0 68261 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.332840] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.335482] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:13 2020][8592438.341578] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.346749] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:13 2020][8592438.354946] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:13 2020][8592438.361217] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:13 2020][8592438.366890] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:13 2020][8592438.372644] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:13 2020][8592438.379349] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:13 2020][8592438.384424] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.390694] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:13 2020][8592438.397315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:13 2020][8592438.403616] lquota_wb_fir-O S ffff9c7a1ef330c0 0 68280 2 0x00000080 [Tue Mar 17 17:59:13 2020][8592438.410893] Call Trace: [Tue Mar 17 17:59:13 2020][8592438.413520] [] schedule+0x29/0x70 [Tue Mar 17 17:59:13 2020][8592438.418661] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:14 2020][8592438.424697] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592438.432528] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:14 2020][8592438.439081] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:14 2020][8592438.445877] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:14 2020][8592438.452496] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592438.458268] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:14 2020][8592438.464628] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.469708] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.475974] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.482587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.488874] inconsistency_v S ffff9c7a32a89040 0 68288 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592438.496146] Call Trace: [Tue Mar 17 17:59:14 2020][8592438.498776] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592438.504878] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592438.510042] [] ofd_inconsistency_verification_main+0x960/0xee0 [ofd] [Tue Mar 17 17:59:14 2020][8592438.518215] [] ? dequeue_task_fair+0x41e/0x660 [Tue Mar 17 17:59:14 2020][8592438.524481] [] ? __switch_to+0xce/0x580 [Tue Mar 17 17:59:14 2020][8592438.530154] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592438.535908] [] ? ofd_cb_soft_sync+0x260/0x260 [ofd] [Tue Mar 17 17:59:14 2020][8592438.542614] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.547690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.553958] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.560594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.566861] lquota_wb_fir-O S ffff9c5a1fef4100 0 68307 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592438.574131] Call Trace: [Tue Mar 17 17:59:14 2020][8592438.576759] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592438.581914] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:14 2020][8592438.587921] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:14 2020][8592438.594453] [] ? qsd_job_pending+0x7e/0x1b0 [lquota] [Tue Mar 17 17:59:14 2020][8592438.601262] [] qsd_upd_thread+0xae6/0xe30 [lquota] [Tue Mar 17 17:59:14 2020][8592438.607880] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592438.613631] [] ? qsd_upd_add+0xf0/0xf0 [lquota] [Tue Mar 17 17:59:14 2020][8592438.620003] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.625058] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.631339] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.637956] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.644230] ll_ost02_008 S ffff9c4b6961b0c0 0 68322 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592438.651532] Call Trace: [Tue Mar 17 17:59:14 2020][8592438.654163] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592438.660263] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592438.665459] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.672335] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592438.678113] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.684599] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.692167] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.697262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.703535] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.710149] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.716430] ll_ost01_008 S ffff9c5a144fa080 0 68323 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592438.723699] Call Trace: [Tue Mar 17 17:59:14 2020][8592438.726329] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592438.732422] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592438.737608] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.744488] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592438.750270] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.756760] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.764348] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.769402] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.775684] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.782298] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.788564] ll_ost03_006 D ffff9c5a144fc100 0 68338 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592438.795851] Call Trace: [Tue Mar 17 17:59:14 2020][8592438.798481] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592438.803641] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:14 2020][8592438.810707] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:14 2020][8592438.816719] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:14 2020][8592438.823973] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:14 2020][8592438.830684] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592438.838182] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:14 2020][8592438.844364] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:14 2020][8592438.851161] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592438.858494] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:14 2020][8592438.866070] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592438.873210] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:14 2020][8592438.879497] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:14 2020][8592438.885601] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:14 2020][8592438.892361] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.900132] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.908144] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:14 2020][8592438.914639] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.921718] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.929466] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:14 2020][8592438.936731] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.944587] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.951552] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:14 2020][8592438.957004] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.963495] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592438.971067] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592438.976136] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.982406] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592438.989026] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592438.995304] ll_ost00_007 S ffff9c5a144f9040 0 68339 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592439.002579] Call Trace: [Tue Mar 17 17:59:14 2020][8592439.005208] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592439.011309] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592439.016500] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.023387] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592439.029165] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.035655] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.043228] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592439.048283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.054570] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592439.061185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.067451] ll_ost00_008 S ffff9c5a144fe180 0 68340 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592439.074737] Call Trace: [Tue Mar 17 17:59:14 2020][8592439.077368] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592439.083467] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592439.088666] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.095544] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:14 2020][8592439.102161] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592439.107951] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.114421] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.121993] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592439.127068] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.133337] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592439.139957] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.146252] ll_ost02_009 D ffff9c5a144fb0c0 0 68341 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592439.153525] Call Trace: [Tue Mar 17 17:59:14 2020][8592439.156157] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592439.161308] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:14 2020][8592439.168373] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:14 2020][8592439.174399] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:14 2020][8592439.181635] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:14 2020][8592439.188345] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592439.195847] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:14 2020][8592439.202033] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:14 2020][8592439.208829] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592439.216173] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:14 2020][8592439.223743] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:14 2020][8592439.230885] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:14 2020][8592439.237174] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:14 2020][8592439.243277] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:14 2020][8592439.250010] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.257799] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.265810] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:14 2020][8592439.272302] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.279385] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.287161] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:14 2020][8592439.294411] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.302257] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:14 2020][8592439.307688] [] ? ptlrpc_server_handle_req_in+0x8df/0xd60 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.315636] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.322125] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.329702] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592439.334776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.341044] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592439.347665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.353968] ll_ost02_010 S ffff9c5a29769040 0 68342 2 0x00000080 [Tue Mar 17 17:59:14 2020][8592439.361259] Call Trace: [Tue Mar 17 17:59:14 2020][8592439.363888] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:14 2020][8592439.369981] [] schedule+0x29/0x70 [Tue Mar 17 17:59:14 2020][8592439.375166] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.382047] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:14 2020][8592439.387833] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.394317] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:14 2020][8592439.401883] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:14 2020][8592439.406965] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:14 2020][8592439.413238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:14 2020][8592439.419859] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.426147] ll_ost_create03 S ffff9c5a2976a080 0 68343 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592439.433420] Call Trace: [Tue Mar 17 17:59:15 2020][8592439.436061] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:15 2020][8592439.442157] [] schedule+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592439.447341] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.454216] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:15 2020][8592439.459996] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.466503] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.474078] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592439.479155] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.485424] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592439.492046] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.498332] ll_ost03_007 D ffff9c5a2976b0c0 0 68344 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592439.505604] Call Trace: [Tue Mar 17 17:59:15 2020][8592439.508250] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:15 2020][8592439.514257] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592439.520893] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:15 2020][8592439.527264] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:15 2020][8592439.532614] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.539631] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.546710] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.554460] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:15 2020][8592439.561725] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.569590] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.576567] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:15 2020][8592439.581999] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.588489] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.596078] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592439.601132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.607422] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592439.614036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.620303] ll_ost02_011 D ffff9c6a01fee180 0 68345 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592439.627587] Call Trace: [Tue Mar 17 17:59:15 2020][8592439.630217] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592439.636839] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:15 2020][8592439.643215] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:15 2020][8592439.648533] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:15 2020][8592439.654033] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.661027] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.668420] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:15 2020][8592439.674349] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:15 2020][8592439.680639] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:15 2020][8592439.685960] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.692926] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.700429] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.707745] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.715407] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.722573] [] evict+0xb4/0x180 [Tue Mar 17 17:59:15 2020][8592439.727546] [] iput+0xfc/0x190 [Tue Mar 17 17:59:15 2020][8592439.732441] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.739794] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:15 2020][8592439.747199] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:15 2020][8592439.754366] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:15 2020][8592439.760984] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:15 2020][8592439.767731] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.775496] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.783502] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:15 2020][8592439.790007] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.797091] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.804837] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:15 2020][8592439.812106] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.819970] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.826933] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:15 2020][8592439.832725] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.839216] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.846796] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592439.851867] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.858136] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592439.864782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.871049] ll_ost03_008 S ffff9c6a017f1040 0 68346 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592439.878325] Call Trace: [Tue Mar 17 17:59:15 2020][8592439.880969] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:15 2020][8592439.887064] [] schedule+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592439.892247] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.899121] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:15 2020][8592439.904904] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.911392] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592439.918958] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592439.924032] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.930301] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592439.936916] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592439.943205] ll_ost01_009 D ffff9c6a017f30c0 0 68347 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592439.950476] Call Trace: [Tue Mar 17 17:59:15 2020][8592439.953106] [] schedule+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592439.958262] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:15 2020][8592439.965331] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:15 2020][8592439.971343] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:15 2020][8592439.978599] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:15 2020][8592439.985311] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592439.992817] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:15 2020][8592439.999006] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.005803] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.013136] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.020710] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.027851] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:15 2020][8592440.034140] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:15 2020][8592440.040243] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:15 2020][8592440.046980] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.054776] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.062785] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:15 2020][8592440.069271] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.076366] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.084117] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:15 2020][8592440.091384] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.099246] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.106210] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:15 2020][8592440.111655] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.118145] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.125712] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592440.130788] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592440.137062] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592440.143690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592440.149961] ll_ost_io02_003 D ffff9c6a017f6180 0 68348 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592440.157254] Call Trace: [Tue Mar 17 17:59:15 2020][8592440.159884] [] schedule+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592440.165027] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.172092] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:15 2020][8592440.178123] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.185378] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:15 2020][8592440.191234] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.197964] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.205272] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:15 2020][8592440.211477] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.218267] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.225585] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.233171] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:15 2020][8592440.240317] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:15 2020][8592440.247111] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:15 2020][8592440.253357] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.260092] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.266742] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.273829] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:15 2020][8592440.281142] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:15 2020][8592440.286895] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:15 2020][8592440.293462] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.300891] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.307970] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.315746] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:15 2020][8592440.322997] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.330855] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.337836] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:15 2020][8592440.343265] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.349757] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:15 2020][8592440.357331] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:15 2020][8592440.362390] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592440.368673] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:15 2020][8592440.375288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:15 2020][8592440.381592] ll_ost_io01_003 D ffff9c7a2da5c100 0 68349 2 0x00000080 [Tue Mar 17 17:59:15 2020][8592440.388864] Call Trace: [Tue Mar 17 17:59:15 2020][8592440.391493] [] schedule+0x29/0x70 [Tue Mar 17 17:59:15 2020][8592440.396637] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:15 2020][8592440.403704] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:16 2020][8592440.409715] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592440.416954] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:16 2020][8592440.422798] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:16 2020][8592440.429520] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.436831] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:16 2020][8592440.443050] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592440.449844] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.457160] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.464746] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.471892] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:16 2020][8592440.478686] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:16 2020][8592440.484919] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.491650] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.498298] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.505364] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:16 2020][8592440.510901] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.517980] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.525743] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:16 2020][8592440.533000] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.540859] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.547839] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:16 2020][8592440.553269] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.559759] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.567334] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:16 2020][8592440.572396] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.578697] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:16 2020][8592440.585316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.591581] ll_ost01_010 S ffff9c7a2da5b0c0 0 68350 2 0x00000080 [Tue Mar 17 17:59:16 2020][8592440.598867] Call Trace: [Tue Mar 17 17:59:16 2020][8592440.601498] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:16 2020][8592440.607599] [] schedule+0x29/0x70 [Tue Mar 17 17:59:16 2020][8592440.612793] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.619670] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:16 2020][8592440.625462] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.631937] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.639539] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:16 2020][8592440.644597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.650871] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:16 2020][8592440.657498] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.663768] ll_ost_io03_003 D ffff9c7a2da58000 0 68351 2 0x00000080 [Tue Mar 17 17:59:16 2020][8592440.671043] Call Trace: [Tue Mar 17 17:59:16 2020][8592440.673675] [] schedule+0x29/0x70 [Tue Mar 17 17:59:16 2020][8592440.678837] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 17:59:16 2020][8592440.686320] [] ? mempool_alloc+0x6e/0x170 [Tue Mar 17 17:59:16 2020][8592440.692162] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:16 2020][8592440.698204] [] raid5_make_request+0x1a5/0xca0 [raid456] [Tue Mar 17 17:59:16 2020][8592440.705258] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 17:59:16 2020][8592440.710405] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:16 2020][8592440.716443] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.724273] [] md_handle_request+0xd0/0x150 [Tue Mar 17 17:59:16 2020][8592440.730288] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 17:59:16 2020][8592440.737442] [] md_make_request+0x79/0x190 [Tue Mar 17 17:59:16 2020][8592440.743278] [] generic_make_request+0x147/0x380 [Tue Mar 17 17:59:16 2020][8592440.749640] [] ? md_mergeable_bvec+0x46/0x50 [Tue Mar 17 17:59:16 2020][8592440.755754] [] submit_bio+0x70/0x150 [Tue Mar 17 17:59:16 2020][8592440.761176] [] ? lprocfs_oh_tally+0x17/0x40 [obdclass] [Tue Mar 17 17:59:16 2020][8592440.768177] [] osd_submit_bio+0x1c/0x60 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.775064] [] osd_do_bio.isra.35+0x9b5/0xab0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.782460] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:16 2020][8592440.788666] [] ? jbd2_journal_add_journal_head+0x65/0x150 [jbd2] [Tue Mar 17 17:59:16 2020][8592440.796501] [] ? recalc_bh_state.part.21+0x70/0x90 [Tue Mar 17 17:59:16 2020][8592440.803122] [] ? jbd2_journal_get_write_access+0x32/0x40 [jbd2] [Tue Mar 17 17:59:16 2020][8592440.810889] [] osd_write_commit+0x3ec/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.818119] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592440.825446] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:16 2020][8592440.832234] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:16 2020][8592440.838481] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.845218] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.851867] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.858947] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:16 2020][8592440.866256] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:16 2020][8592440.872003] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:16 2020][8592440.877539] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.884618] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.892366] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:16 2020][8592440.899648] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.907508] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.914469] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:16 2020][8592440.919913] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.926398] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.933972] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:16 2020][8592440.939046] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.945315] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:16 2020][8592440.951927] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592440.958206] ll_ost03_009 S ffff9c7a2da5e180 0 68352 2 0x00000080 [Tue Mar 17 17:59:16 2020][8592440.965497] Call Trace: [Tue Mar 17 17:59:16 2020][8592440.968126] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:16 2020][8592440.974229] [] schedule+0x29/0x70 [Tue Mar 17 17:59:16 2020][8592440.979421] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.986301] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:16 2020][8592440.992084] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592440.998573] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.006147] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:16 2020][8592441.011223] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592441.017490] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:16 2020][8592441.024104] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592441.030400] ll_ost_io00_003 D ffff9c7a2da5d140 0 68353 2 0x00000080 [Tue Mar 17 17:59:16 2020][8592441.037673] Call Trace: [Tue Mar 17 17:59:16 2020][8592441.040303] [] schedule+0x29/0x70 [Tue Mar 17 17:59:16 2020][8592441.045456] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.052520] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:16 2020][8592441.058532] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.065773] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:16 2020][8592441.071609] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.078336] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.085647] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:16 2020][8592441.091864] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.098659] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.105980] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.113573] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.120718] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:16 2020][8592441.127510] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:16 2020][8592441.133742] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.140447] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:16 2020][8592441.145973] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.153081] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.160828] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:16 2020][8592441.168079] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.175969] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.182930] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:16 2020][8592441.188359] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.194847] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.202413] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:16 2020][8592441.207495] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592441.213768] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:16 2020][8592441.220387] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:16 2020][8592441.226682] ll_ost_io02_004 D ffff9c7a2da59040 0 68354 2 0x00000080 [Tue Mar 17 17:59:16 2020][8592441.233966] Call Trace: [Tue Mar 17 17:59:16 2020][8592441.236604] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.244266] [] schedule+0x29/0x70 [Tue Mar 17 17:59:16 2020][8592441.249431] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.256486] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:16 2020][8592441.262502] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.269745] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.276451] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.284221] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:16 2020][8592441.290411] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:16 2020][8592441.297208] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.304539] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.312114] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:16 2020][8592441.319269] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:16 2020][8592441.325546] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:16 2020][8592441.332354] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:16 2020][8592441.338575] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.345307] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.351980] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.359054] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:16 2020][8592441.366380] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:16 2020][8592441.372133] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:16 2020][8592441.377654] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.384745] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.392497] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:16 2020][8592441.399763] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.407620] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.414585] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:16 2020][8592441.420049] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:16 2020][8592441.426520] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.434094] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592441.439166] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.445437] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592441.452059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.458339] ll_ost_io01_004 D ffff9c7a2f4e1040 0 68355 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592441.465609] Call Trace: [Tue Mar 17 17:59:17 2020][8592441.468254] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592441.473400] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:17 2020][8592441.480479] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:17 2020][8592441.486494] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:17 2020][8592441.493733] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:17 2020][8592441.499573] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:17 2020][8592441.506301] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.513610] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:17 2020][8592441.519817] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:17 2020][8592441.526617] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.533934] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.541526] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.548686] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:17 2020][8592441.555483] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:17 2020][8592441.561714] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.568449] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.575096] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.582178] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:17 2020][8592441.589487] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:17 2020][8592441.595234] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:17 2020][8592441.600775] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.607857] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.615634] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:17 2020][8592441.622884] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.630744] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.637723] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:17 2020][8592441.643156] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.649642] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.657210] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592441.662284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.668553] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592441.675166] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.681461] ll_ost03_010 S ffff9c7a2f4e2080 0 68356 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592441.688735] Call Trace: [Tue Mar 17 17:59:17 2020][8592441.691366] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:17 2020][8592441.697465] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592441.702648] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.709522] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:17 2020][8592441.715304] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.721797] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.729366] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592441.734433] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.740705] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592441.747352] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.753626] ll_ost_io02_005 D ffff9c7a2f4e4100 0 68357 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592441.760895] Call Trace: [Tue Mar 17 17:59:17 2020][8592441.763537] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592441.768687] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.775757] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:17 2020][8592441.782894] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:17 2020][8592441.788912] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:17 2020][8592441.795114] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:17 2020][8592441.801926] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:17 2020][8592441.808147] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.814863] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:17 2020][8592441.820897] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.828331] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.835426] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.843175] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:17 2020][8592441.850428] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.858304] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.865271] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:17 2020][8592441.870708] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.877208] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.884779] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592441.889831] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.896112] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592441.902728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.909008] ll_ost02_012 S ffff9c7a2f4e5140 0 68358 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592441.916280] Call Trace: [Tue Mar 17 17:59:17 2020][8592441.918910] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:17 2020][8592441.925012] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592441.930217] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.937091] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:17 2020][8592441.942879] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.949363] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592441.956929] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592441.961996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.968265] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592441.974878] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592441.981158] ll_ost_io01_005 D ffff9c7a2f4e30c0 0 68359 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592441.988431] Call Trace: [Tue Mar 17 17:59:17 2020][8592441.991060] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592441.996213] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.003298] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.010437] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:17 2020][8592442.016449] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:17 2020][8592442.022651] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:17 2020][8592442.029448] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:17 2020][8592442.035665] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.042415] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.049061] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.056132] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:17 2020][8592442.063457] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:17 2020][8592442.069207] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:17 2020][8592442.075774] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.083205] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.090289] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.098054] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:17 2020][8592442.105310] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.113183] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.120151] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:17 2020][8592442.125588] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.132091] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.139662] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592442.144720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592442.151001] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592442.157618] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592442.163890] ll_ost_io00_004 D ffff9c7a14790000 0 68360 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592442.171176] Call Trace: [Tue Mar 17 17:59:17 2020][8592442.173805] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592442.178967] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:17 2020][8592442.186046] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:17 2020][8592442.192077] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:17 2020][8592442.199317] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:17 2020][8592442.205158] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:17 2020][8592442.211886] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.219194] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:17 2020][8592442.225398] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:17 2020][8592442.232189] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.239509] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.247103] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:17 2020][8592442.254261] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:17 2020][8592442.261059] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:17 2020][8592442.267289] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.274024] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.280686] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.287758] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:17 2020][8592442.295086] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:17 2020][8592442.300836] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:17 2020][8592442.307410] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.314854] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.321935] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.329711] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:17 2020][8592442.336962] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.344844] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.351814] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:17 2020][8592442.357302] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.363775] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.371361] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:17 2020][8592442.376417] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592442.382692] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:17 2020][8592442.389342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:17 2020][8592442.395615] ll_ost03_011 S ffff9c7a14792080 0 68361 2 0x00000080 [Tue Mar 17 17:59:17 2020][8592442.402889] Call Trace: [Tue Mar 17 17:59:17 2020][8592442.405532] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:17 2020][8592442.411628] [] schedule+0x29/0x70 [Tue Mar 17 17:59:17 2020][8592442.416815] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:17 2020][8592442.423693] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:17 2020][8592442.429482] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.435953] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.443537] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592442.448594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.454898] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592442.461515] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.467781] ll_ost_io01_006 D ffff9c7a14796180 0 68362 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592442.475068] Call Trace: [Tue Mar 17 17:59:18 2020][8592442.477696] [] schedule+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592442.482849] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:18 2020][8592442.489915] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:18 2020][8592442.495927] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:18 2020][8592442.503164] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:18 2020][8592442.509002] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:18 2020][8592442.515731] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.523040] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:18 2020][8592442.529222] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:18 2020][8592442.536036] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.543354] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.550942] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.558084] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:18 2020][8592442.564879] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:18 2020][8592442.571112] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.577860] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.584508] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.591593] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:18 2020][8592442.598903] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:18 2020][8592442.604657] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:18 2020][8592442.610198] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.617280] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.625026] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:18 2020][8592442.632294] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.640156] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.647150] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:18 2020][8592442.652585] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.659060] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.666645] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592442.671700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.677982] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592442.684598] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.690870] ll_ost_io02_006 D ffff9c7a14795140 0 68363 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592442.698159] Call Trace: [Tue Mar 17 17:59:18 2020][8592442.700787] [] schedule+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592442.705942] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.713024] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.720156] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:18 2020][8592442.726166] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:18 2020][8592442.732371] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:18 2020][8592442.739167] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:18 2020][8592442.745386] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.752136] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.758778] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.765846] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:18 2020][8592442.771380] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.778463] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.786224] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:18 2020][8592442.793481] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.801354] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.808316] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:18 2020][8592442.813752] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.820239] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.827808] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592442.832883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.839166] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592442.845781] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592442.852068] ll_ost_io00_005 D ffff9c7a14794100 0 68364 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592442.859342] Call Trace: [Tue Mar 17 17:59:18 2020][8592442.861971] [] schedule+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592442.867126] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.874195] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592442.881332] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:18 2020][8592442.887350] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:18 2020][8592442.893554] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:18 2020][8592442.900365] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:18 2020][8592442.906585] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.913331] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.919982] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.927045] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:18 2020][8592442.933080] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.940521] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.947605] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.955351] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:18 2020][8592442.962619] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.970479] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.977473] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:18 2020][8592442.982908] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.989381] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592442.996969] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592443.002024] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.008306] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592443.014922] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.021194] ll_ost_io03_004 D ffff9c6ae9a1b0c0 0 68365 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592443.028480] Call Trace: [Tue Mar 17 17:59:18 2020][8592443.031113] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:18 2020][8592443.036432] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592443.043079] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:18 2020][8592443.049442] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:18 2020][8592443.054764] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:18 2020][8592443.060279] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.067250] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.074660] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:18 2020][8592443.080935] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.087830] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.096203] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.103433] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.111100] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:18 2020][8592443.116683] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.124344] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.132281] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.139686] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.146741] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.154933] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:18 2020][8592443.161906] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.169132] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:18 2020][8592443.176477] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:18 2020][8592443.183282] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:18 2020][8592443.189503] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.196261] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.202905] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.209956] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:18 2020][8592443.215489] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.222571] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.230320] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:18 2020][8592443.237601] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.245459] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.252425] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:18 2020][8592443.257874] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.264344] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.271916] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592443.276990] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.283260] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592443.289879] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.296162] ll_ost03_012 S ffff9c7a3693d140 0 68366 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592443.303451] Call Trace: [Tue Mar 17 17:59:18 2020][8592443.306078] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:18 2020][8592443.312172] [] schedule+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592443.317354] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.324227] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:18 2020][8592443.330002] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.336492] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:18 2020][8592443.344067] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:18 2020][8592443.349127] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.355426] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:18 2020][8592443.362040] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:18 2020][8592443.368341] ll_ost01_011 D ffff9c7a3693a080 0 68367 2 0x00000080 [Tue Mar 17 17:59:18 2020][8592443.375610] Call Trace: [Tue Mar 17 17:59:18 2020][8592443.378239] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:18 2020][8592443.383560] [] ? bh_lru_install+0x18a/0x1e0 [Tue Mar 17 17:59:18 2020][8592443.389579] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:18 2020][8592443.394898] [] schedule+0x29/0x70 [Tue Mar 17 17:59:18 2020][8592443.400044] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:18 2020][8592443.406072] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:18 2020][8592443.411388] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:18 2020][8592443.417568] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:18 2020][8592443.422990] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:18 2020][8592443.428392] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:18 2020][8592443.433966] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:19 2020][8592443.439299] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:19 2020][8592443.445743] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.451843] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.458741] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.467006] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.474600] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.482867] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.489866] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.497611] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.504491] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.511646] [] evict+0xb4/0x180 [Tue Mar 17 17:59:19 2020][8592443.516620] [] iput+0xfc/0x190 [Tue Mar 17 17:59:19 2020][8592443.521505] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.528852] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:19 2020][8592443.536257] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:19 2020][8592443.543410] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:19 2020][8592443.550062] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:19 2020][8592443.556792] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.564566] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.572591] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:19 2020][8592443.579070] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.586151] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.593912] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:19 2020][8592443.601167] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.609044] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.616009] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:19 2020][8592443.621439] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.627942] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.635511] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:19 2020][8592443.640572] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.646861] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:19 2020][8592443.653476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.659743] ll_ost_io01_007 D ffff9c7a36939040 0 68368 2 0x00000080 [Tue Mar 17 17:59:19 2020][8592443.667037] Call Trace: [Tue Mar 17 17:59:19 2020][8592443.669664] [] schedule+0x29/0x70 [Tue Mar 17 17:59:19 2020][8592443.674812] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.681888] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:19 2020][8592443.687905] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.695143] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:19 2020][8592443.700982] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.707710] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.715021] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:19 2020][8592443.721224] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:19 2020][8592443.728015] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.735334] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.742928] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.750072] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:19 2020][8592443.756893] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:19 2020][8592443.763110] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.769820] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:19 2020][8592443.775361] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.782442] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.790193] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:19 2020][8592443.797457] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.805320] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.812301] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:19 2020][8592443.817748] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.824222] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.831813] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:19 2020][8592443.836874] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.843162] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:19 2020][8592443.849779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.856053] ll_ost01_012 S ffff9c7a20e86180 0 68369 2 0x00000080 [Tue Mar 17 17:59:19 2020][8592443.863346] Call Trace: [Tue Mar 17 17:59:19 2020][8592443.865977] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:19 2020][8592443.872076] [] schedule+0x29/0x70 [Tue Mar 17 17:59:19 2020][8592443.877276] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.884150] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:19 2020][8592443.889924] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.896414] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592443.903988] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:19 2020][8592443.909048] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.915332] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:19 2020][8592443.921946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592443.928221] ll_ost_io02_007 D ffff9c7a20e81040 0 68370 2 0x00000080 [Tue Mar 17 17:59:19 2020][8592443.935507] Call Trace: [Tue Mar 17 17:59:19 2020][8592443.938135] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:19 2020][8592443.944758] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:19 2020][8592443.951144] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:19 2020][8592443.956466] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:19 2020][8592443.961968] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.968948] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.976347] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:19 2020][8592443.982636] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 17:59:19 2020][8592443.987788] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592443.995019] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.002192] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 17:59:19 2020][8592444.009074] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:19 2020][8592444.014661] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.022336] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.030002] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 17:59:19 2020][8592444.036535] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.043951] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.051010] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.059195] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.066180] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.073410] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.080754] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:19 2020][8592444.087545] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:19 2020][8592444.093762] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.100506] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.107150] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.114200] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:19 2020][8592444.119741] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.126826] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.134577] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:19 2020][8592444.141862] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.149718] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.156699] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:19 2020][8592444.162129] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.168618] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.176185] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:19 2020][8592444.181246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592444.187548] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:19 2020][8592444.194167] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592444.200447] ll_ost_io00_006 D ffff9c7a20e830c0 0 68371 2 0x00000080 [Tue Mar 17 17:59:19 2020][8592444.207735] Call Trace: [Tue Mar 17 17:59:19 2020][8592444.210387] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.218045] [] schedule+0x29/0x70 [Tue Mar 17 17:59:19 2020][8592444.223211] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.230278] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:19 2020][8592444.236289] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.243533] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.250238] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.257995] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:19 2020][8592444.264179] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.270994] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.278338] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.285910] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:19 2020][8592444.293077] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:19 2020][8592444.299350] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:19 2020][8592444.306145] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:19 2020][8592444.312378] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.319083] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:19 2020][8592444.325121] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.332564] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.339660] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.347410] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:19 2020][8592444.354676] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.362562] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.369545] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:19 2020][8592444.374976] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.381464] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:19 2020][8592444.389030] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:19 2020][8592444.394083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592444.400371] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:19 2020][8592444.406987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:19 2020][8592444.413280] ll_ost_io01_008 D ffff9c79caef9040 0 68372 2 0x00000080 [Tue Mar 17 17:59:19 2020][8592444.420555] Call Trace: [Tue Mar 17 17:59:19 2020][8592444.423186] [] schedule+0x29/0x70 [Tue Mar 17 17:59:19 2020][8592444.428337] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:19 2020][8592444.435402] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:20 2020][8592444.441416] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.448654] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:20 2020][8592444.454493] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.461221] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.468530] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:20 2020][8592444.474751] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.481545] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.488861] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.496448] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.503592] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:20 2020][8592444.510384] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:20 2020][8592444.516618] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.523323] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:20 2020][8592444.529365] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.536830] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.543916] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.551681] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:20 2020][8592444.558936] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.566812] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.573779] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:20 2020][8592444.579230] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.585698] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.593283] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:20 2020][8592444.598342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592444.604644] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:20 2020][8592444.611262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592444.617526] ll_ost_io00_007 D ffff9c79caefe180 0 68373 2 0x00000080 [Tue Mar 17 17:59:20 2020][8592444.624816] Call Trace: [Tue Mar 17 17:59:20 2020][8592444.627443] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:20 2020][8592444.634064] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:20 2020][8592444.640431] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:20 2020][8592444.645746] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:20 2020][8592444.651241] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.658223] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.665622] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:20 2020][8592444.671669] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:20 2020][8592444.678028] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:20 2020][8592444.684649] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.691371] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:20 2020][8592444.698511] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:20 2020][8592444.703745] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:20 2020][8592444.709426] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.716830] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.723818] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.731137] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.738047] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.745359] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:20 2020][8592444.752240] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:20 2020][8592444.758472] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.765178] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 17:59:20 2020][8592444.770843] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:20 2020][8592444.776612] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:20 2020][8592444.782135] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.789212] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.796989] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:20 2020][8592444.804238] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.812101] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.819079] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:20 2020][8592444.824512] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.830986] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.838569] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:20 2020][8592444.843624] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592444.849907] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:20 2020][8592444.856522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592444.862788] ll_ost_io03_005 D ffff9c79caef8000 0 68374 2 0x00000080 [Tue Mar 17 17:59:20 2020][8592444.870083] Call Trace: [Tue Mar 17 17:59:20 2020][8592444.872711] [] schedule+0x29/0x70 [Tue Mar 17 17:59:20 2020][8592444.877855] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.884919] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:20 2020][8592444.890932] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.898172] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:20 2020][8592444.904010] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.910739] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.918047] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:20 2020][8592444.924257] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592444.931051] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.938368] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.945955] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592444.953098] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:20 2020][8592444.959895] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:20 2020][8592444.966126] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.972860] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.979523] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.986575] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:20 2020][8592444.992130] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592444.999216] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.006964] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:20 2020][8592445.014231] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.022094] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.029074] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:20 2020][8592445.034504] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.040981] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.048566] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:20 2020][8592445.053620] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.059915] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:20 2020][8592445.066535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.072812] ll_ost_io02_008 D ffff9c79caefd140 0 68375 2 0x00000080 [Tue Mar 17 17:59:20 2020][8592445.080102] Call Trace: [Tue Mar 17 17:59:20 2020][8592445.082732] [] schedule+0x29/0x70 [Tue Mar 17 17:59:20 2020][8592445.087879] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.094948] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.102083] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:20 2020][8592445.108111] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:20 2020][8592445.114299] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:20 2020][8592445.121096] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:20 2020][8592445.127341] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.134077] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.140726] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.147791] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:20 2020][8592445.153312] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.160407] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.168157] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:20 2020][8592445.175423] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.183286] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.190296] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:20 2020][8592445.195734] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.202206] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.209814] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:20 2020][8592445.214875] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.221150] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:20 2020][8592445.227799] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.234071] ll_ost00_009 S ffff9c79caefa080 0 68376 2 0x00000080 [Tue Mar 17 17:59:20 2020][8592445.241365] Call Trace: [Tue Mar 17 17:59:20 2020][8592445.243995] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:20 2020][8592445.250097] [] schedule+0x29/0x70 [Tue Mar 17 17:59:20 2020][8592445.255302] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.262178] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:20 2020][8592445.267960] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.274449] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:20 2020][8592445.282015] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:20 2020][8592445.287067] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.293347] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:20 2020][8592445.299966] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:20 2020][8592445.306238] ll_ost_io00_008 D ffff9c79caefc100 0 68377 2 0x00000080 [Tue Mar 17 17:59:20 2020][8592445.313541] Call Trace: [Tue Mar 17 17:59:20 2020][8592445.316180] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.323842] [] schedule+0x29/0x70 [Tue Mar 17 17:59:20 2020][8592445.329008] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:20 2020][8592445.336062] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:20 2020][8592445.342077] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592445.349323] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:20 2020][8592445.356035] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.363792] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:20 2020][8592445.369976] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:20 2020][8592445.376772] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.384119] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.391699] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:20 2020][8592445.398837] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:20 2020][8592445.405126] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:20 2020][8592445.411925] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:21 2020][8592445.418141] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.424891] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.431539] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.438612] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:21 2020][8592445.445951] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:21 2020][8592445.451701] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:21 2020][8592445.458265] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.465713] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.472793] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.480555] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:21 2020][8592445.487812] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.495683] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.502646] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:21 2020][8592445.508080] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.514585] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.522152] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:21 2020][8592445.527207] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.533495] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:21 2020][8592445.540111] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.546398] ldlm_cn02_003 S ffff9c79caefb0c0 0 68378 2 0x00000080 [Tue Mar 17 17:59:21 2020][8592445.553671] Call Trace: [Tue Mar 17 17:59:21 2020][8592445.556313] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:21 2020][8592445.562409] [] schedule+0x29/0x70 [Tue Mar 17 17:59:21 2020][8592445.567593] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.574489] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:21 2020][8592445.580265] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.586754] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.594321] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:21 2020][8592445.599395] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.605665] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:21 2020][8592445.612280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.618567] ll_ost_io03_006 D ffff9c5a153f1040 0 68379 2 0x00000080 [Tue Mar 17 17:59:21 2020][8592445.625839] Call Trace: [Tue Mar 17 17:59:21 2020][8592445.628480] [] schedule+0x29/0x70 [Tue Mar 17 17:59:21 2020][8592445.633629] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.640707] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:21 2020][8592445.646721] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.653965] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.660683] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.668011] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:21 2020][8592445.674196] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.680994] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.688324] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.695899] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.703040] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:21 2020][8592445.709863] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:21 2020][8592445.716078] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.722780] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:21 2020][8592445.728839] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.736272] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.743363] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.751129] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:21 2020][8592445.758386] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.766261] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.773245] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:21 2020][8592445.778676] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.785161] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.792729] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:21 2020][8592445.797803] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.804082] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:21 2020][8592445.810718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592445.816986] ll_ost_io01_009 D ffff9c5a153f30c0 0 68380 2 0x00000080 [Tue Mar 17 17:59:21 2020][8592445.824272] Call Trace: [Tue Mar 17 17:59:21 2020][8592445.826902] [] schedule+0x29/0x70 [Tue Mar 17 17:59:21 2020][8592445.832053] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.839132] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:21 2020][8592445.845147] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.852386] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:21 2020][8592445.858226] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.864955] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.872264] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:21 2020][8592445.878469] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592445.885270] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.892598] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.900174] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592445.907343] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:21 2020][8592445.914135] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:21 2020][8592445.920352] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.927098] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.933750] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.940825] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:21 2020][8592445.948167] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:21 2020][8592445.953920] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:21 2020][8592445.960472] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.967924] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.975002] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.982752] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:21 2020][8592445.990018] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592445.997879] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.004847] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:21 2020][8592446.010296] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.016765] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.024353] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:21 2020][8592446.029407] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592446.035674] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:21 2020][8592446.042324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592446.048597] ll_ost_io02_009 D ffff9c5a153f5140 0 68381 2 0x00000080 [Tue Mar 17 17:59:21 2020][8592446.055872] Call Trace: [Tue Mar 17 17:59:21 2020][8592446.058514] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:21 2020][8592446.065132] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:21 2020][8592446.071508] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:21 2020][8592446.076823] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:21 2020][8592446.082317] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.089299] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.096697] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:21 2020][8592446.102731] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:21 2020][8592446.109097] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:21 2020][8592446.115710] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:21 2020][8592446.121210] [] ? dqget+0x1a7/0x450 [Tue Mar 17 17:59:21 2020][8592446.126442] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:21 2020][8592446.133582] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:21 2020][8592446.138831] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:21 2020][8592446.144498] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.151912] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592446.158875] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.166218] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.173097] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.180412] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:21 2020][8592446.187322] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:21 2020][8592446.193538] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.200275] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.206938] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.214026] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:21 2020][8592446.221348] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:21 2020][8592446.227094] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:21 2020][8592446.232642] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.239718] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.247468] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:21 2020][8592446.254734] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.262599] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.269563] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:21 2020][8592446.275015] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.281492] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:21 2020][8592446.289064] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:21 2020][8592446.294152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592446.300425] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:21 2020][8592446.307046] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:21 2020][8592446.313327] ll_ost_io00_009 D ffff9c5a153f6180 0 68382 2 0x00000080 [Tue Mar 17 17:59:21 2020][8592446.320598] Call Trace: [Tue Mar 17 17:59:21 2020][8592446.323228] [] schedule+0x29/0x70 [Tue Mar 17 17:59:21 2020][8592446.328380] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:21 2020][8592446.335443] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:21 2020][8592446.341454] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592446.348699] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:21 2020][8592446.355413] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.363183] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:21 2020][8592446.369371] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:21 2020][8592446.376169] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.383499] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.391077] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:21 2020][8592446.398216] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:21 2020][8592446.404504] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:21 2020][8592446.411304] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:21 2020][8592446.417521] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.424297] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.431723] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.438803] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.446564] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:22 2020][8592446.453821] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.461694] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.468654] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:22 2020][8592446.474083] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.480572] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.488147] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:22 2020][8592446.493207] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592446.499502] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:22 2020][8592446.506120] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592446.512386] ll_ost_io00_010 D ffff9c5a153f4100 0 68383 2 0x00000080 [Tue Mar 17 17:59:22 2020][8592446.519673] Call Trace: [Tue Mar 17 17:59:22 2020][8592446.522305] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:22 2020][8592446.527622] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:22 2020][8592446.534250] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:22 2020][8592446.540607] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:22 2020][8592446.545926] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:22 2020][8592446.551435] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.558404] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.565801] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:22 2020][8592446.572097] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.580448] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 17:59:22 2020][8592446.586818] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.594066] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.601723] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:22 2020][8592446.607304] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.614981] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.622919] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.630322] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.637391] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.645572] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592446.652544] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.659784] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.667100] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:22 2020][8592446.673909] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:22 2020][8592446.680130] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.686861] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.693536] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.700585] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:22 2020][8592446.706624] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.714065] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.721146] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.728909] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:22 2020][8592446.736166] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.744026] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.751005] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:22 2020][8592446.756436] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.762910] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.770510] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:22 2020][8592446.775569] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592446.781837] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:22 2020][8592446.788470] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592446.794739] ll_ost_io03_007 D ffff9c5a153f0000 0 68384 2 0x00000080 [Tue Mar 17 17:59:22 2020][8592446.802025] Call Trace: [Tue Mar 17 17:59:22 2020][8592446.804653] [] schedule+0x29/0x70 [Tue Mar 17 17:59:22 2020][8592446.809797] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:22 2020][8592446.816862] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:22 2020][8592446.822874] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592446.830131] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:22 2020][8592446.836842] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.844597] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:22 2020][8592446.850782] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592446.857578] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.864910] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.872487] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592446.879627] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:22 2020][8592446.885913] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:22 2020][8592446.892720] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:22 2020][8592446.898940] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.905684] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.912337] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.919385] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:22 2020][8592446.925431] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.932860] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.939939] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.947700] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:22 2020][8592446.954956] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.962850] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.969817] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:22 2020][8592446.975254] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.981745] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592446.989317] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:22 2020][8592446.994370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592447.000659] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:22 2020][8592447.007273] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592447.013543] ll_ost_io01_010 D ffff9c597d2c0000 0 68385 2 0x00000080 [Tue Mar 17 17:59:22 2020][8592447.020834] Call Trace: [Tue Mar 17 17:59:22 2020][8592447.023463] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:22 2020][8592447.030078] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:22 2020][8592447.036466] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:22 2020][8592447.041786] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:22 2020][8592447.047290] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.054279] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.061677] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:22 2020][8592447.067972] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.075202] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.082859] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:22 2020][8592447.088475] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.096143] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 17:59:22 2020][8592447.102686] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.110100] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.117160] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.125350] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592447.132325] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.139551] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.146878] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:22 2020][8592447.153676] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:22 2020][8592447.159897] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.166662] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.173310] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.180379] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:22 2020][8592447.187704] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:22 2020][8592447.193467] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:22 2020][8592447.200035] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.207468] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.214569] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.222316] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:22 2020][8592447.229613] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.237473] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.244439] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:22 2020][8592447.249891] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.256367] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.263941] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:22 2020][8592447.269015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592447.275284] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:22 2020][8592447.281898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:22 2020][8592447.288186] ll_ost_io02_010 D ffff9c597d2c1040 0 68386 2 0x00000080 [Tue Mar 17 17:59:22 2020][8592447.295475] Call Trace: [Tue Mar 17 17:59:22 2020][8592447.298111] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.305766] [] schedule+0x29/0x70 [Tue Mar 17 17:59:22 2020][8592447.310931] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:22 2020][8592447.317976] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:22 2020][8592447.323998] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592447.331222] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:22 2020][8592447.337934] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.345691] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:22 2020][8592447.351874] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:22 2020][8592447.358672] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.366035] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.373614] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:22 2020][8592447.380753] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:22 2020][8592447.387043] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:22 2020][8592447.393840] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:22 2020][8592447.400057] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.406774] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:22 2020][8592447.412295] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:22 2020][8592447.419390] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.427152] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:23 2020][8592447.434409] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.442282] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.449242] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:23 2020][8592447.454688] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.461161] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.468734] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:23 2020][8592447.473799] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.480069] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:23 2020][8592447.486682] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.492978] ll_ost_io00_011 D ffff9c597d2c2080 0 68387 2 0x00000080 [Tue Mar 17 17:59:23 2020][8592447.500253] Call Trace: [Tue Mar 17 17:59:23 2020][8592447.502880] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:23 2020][8592447.509495] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:23 2020][8592447.515872] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:23 2020][8592447.521186] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:23 2020][8592447.526688] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.533671] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.541068] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:23 2020][8592447.547086] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:23 2020][8592447.553476] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:23 2020][8592447.560096] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.566801] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:23 2020][8592447.573954] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:23 2020][8592447.579183] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:23 2020][8592447.584869] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.592274] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.599243] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.606573] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.613457] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.620769] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:23 2020][8592447.627680] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:23 2020][8592447.633895] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.640627] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.647288] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.654342] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:23 2020][8592447.659880] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.666963] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.674728] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:23 2020][8592447.681981] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.689845] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.696844] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:23 2020][8592447.702280] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.708754] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.716338] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:23 2020][8592447.721395] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.727675] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:23 2020][8592447.734292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.740565] ll_ost_io00_012 D ffff9c597d2c30c0 0 68388 2 0x00000080 [Tue Mar 17 17:59:23 2020][8592447.747853] Call Trace: [Tue Mar 17 17:59:23 2020][8592447.750489] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.758142] [] schedule+0x29/0x70 [Tue Mar 17 17:59:23 2020][8592447.763321] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.770371] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:23 2020][8592447.776390] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.783630] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.790335] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.798093] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:23 2020][8592447.804276] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.811076] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.818419] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.825993] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592447.833131] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:23 2020][8592447.839419] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:23 2020][8592447.846219] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:23 2020][8592447.852434] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.859180] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.865834] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.872887] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:23 2020][8592447.878935] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.886364] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.893471] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.901216] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:23 2020][8592447.908469] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.916347] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.923312] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:23 2020][8592447.928741] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.935229] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592447.942796] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:23 2020][8592447.947870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.954155] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:23 2020][8592447.960769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592447.967050] ll_ost_io00_013 D ffff9c597d2c4100 0 68389 2 0x00000080 [Tue Mar 17 17:59:23 2020][8592447.974323] Call Trace: [Tue Mar 17 17:59:23 2020][8592447.976952] [] schedule+0x29/0x70 [Tue Mar 17 17:59:23 2020][8592447.982103] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:23 2020][8592447.989167] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:23 2020][8592447.995178] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592448.002420] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:23 2020][8592448.009128] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.016898] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:23 2020][8592448.023086] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592448.029884] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.037215] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.044790] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.051932] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:23 2020][8592448.058220] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 17:59:23 2020][8592448.064756] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 17:59:23 2020][8592448.071063] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.078153] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.085904] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:23 2020][8592448.093187] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.101044] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.108023] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:23 2020][8592448.113455] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.119929] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.127513] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:23 2020][8592448.132571] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592448.138858] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:23 2020][8592448.145475] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592448.151740] ll_ost_io03_008 D ffff9c597d2c5140 0 68390 2 0x00000080 [Tue Mar 17 17:59:23 2020][8592448.159037] Call Trace: [Tue Mar 17 17:59:23 2020][8592448.161666] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:23 2020][8592448.168286] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:23 2020][8592448.174654] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:23 2020][8592448.179969] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:23 2020][8592448.185472] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.192467] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.199869] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:23 2020][8592448.205888] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:23 2020][8592448.212276] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:23 2020][8592448.218894] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:23 2020][8592448.224386] [] ? dqget+0x1a7/0x450 [Tue Mar 17 17:59:23 2020][8592448.229626] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:23 2020][8592448.236761] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:23 2020][8592448.241987] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:23 2020][8592448.247668] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.255071] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:23 2020][8592448.262041] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.269368] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.276254] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:23 2020][8592448.283595] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:23 2020][8592448.290474] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:23 2020][8592448.296692] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.303436] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.310086] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.317161] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:23 2020][8592448.324485] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:23 2020][8592448.330233] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:23 2020][8592448.335752] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.342867] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.350615] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:23 2020][8592448.357868] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.365741] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.372729] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:23 2020][8592448.378178] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.384654] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:23 2020][8592448.392240] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:23 2020][8592448.397297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592448.403586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:23 2020][8592448.410202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:23 2020][8592448.416503] ll_ost_io02_011 D ffff9c597d2c6180 0 68391 2 0x00000080 [Tue Mar 17 17:59:24 2020][8592448.423779] Call Trace: [Tue Mar 17 17:59:24 2020][8592448.426407] [] schedule+0x29/0x70 [Tue Mar 17 17:59:24 2020][8592448.431549] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.438615] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:24 2020][8592448.444629] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.451871] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.458583] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.466340] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:24 2020][8592448.472541] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.479333] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.486662] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.494240] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.501394] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:24 2020][8592448.507670] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 17:59:24 2020][8592448.514206] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 17:59:24 2020][8592448.520525] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.527606] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.535352] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:24 2020][8592448.542633] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.550492] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.557457] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:24 2020][8592448.562899] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.569369] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.576940] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:24 2020][8592448.582016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592448.588285] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:24 2020][8592448.594907] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592448.601196] ll_ost_io01_011 D ffff9c7a3714e180 0 68392 2 0x00000080 [Tue Mar 17 17:59:24 2020][8592448.608484] Call Trace: [Tue Mar 17 17:59:24 2020][8592448.611113] [] schedule+0x29/0x70 [Tue Mar 17 17:59:24 2020][8592448.616256] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.623321] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:24 2020][8592448.629332] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.636575] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.643279] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.651037] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:24 2020][8592448.657223] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.664020] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.671363] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.678935] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.686076] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:24 2020][8592448.692363] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:24 2020][8592448.699153] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:24 2020][8592448.705371] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.712130] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.718774] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.725857] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:24 2020][8592448.733166] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:24 2020][8592448.738915] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:24 2020][8592448.745487] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.752915] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.759995] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.767756] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:24 2020][8592448.775013] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.782886] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.789847] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:24 2020][8592448.795275] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.801766] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.809353] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:24 2020][8592448.814406] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592448.820689] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:24 2020][8592448.827304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592448.833569] ll_ost_io02_012 D ffff9c7a3714a080 0 68393 2 0x00000080 [Tue Mar 17 17:59:24 2020][8592448.840856] Call Trace: [Tue Mar 17 17:59:24 2020][8592448.843493] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.851154] [] schedule+0x29/0x70 [Tue Mar 17 17:59:24 2020][8592448.856311] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.863360] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:24 2020][8592448.869376] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.876630] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.883343] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.891100] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:24 2020][8592448.897281] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592448.904094] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.911413] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.919001] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592448.926144] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:24 2020][8592448.932418] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:24 2020][8592448.939241] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:24 2020][8592448.945456] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.952186] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.958855] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.965902] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:24 2020][8592448.971438] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.978517] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592448.986266] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:24 2020][8592448.993533] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.001398] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.008389] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:24 2020][8592449.013827] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.020311] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.027878] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:24 2020][8592449.032931] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592449.039214] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:24 2020][8592449.045826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592449.052106] ll_ost_io03_009 D ffff9c7a37148000 0 68394 2 0x00000080 [Tue Mar 17 17:59:24 2020][8592449.059378] Call Trace: [Tue Mar 17 17:59:24 2020][8592449.062009] [] schedule+0x29/0x70 [Tue Mar 17 17:59:24 2020][8592449.067160] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:24 2020][8592449.074237] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:24 2020][8592449.080255] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592449.087496] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:24 2020][8592449.093333] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:24 2020][8592449.100060] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.107371] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:24 2020][8592449.113575] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:24 2020][8592449.120368] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.127684] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.135287] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.142430] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:24 2020][8592449.149225] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:24 2020][8592449.155461] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.162163] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 17:59:24 2020][8592449.167829] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:24 2020][8592449.173600] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:24 2020][8592449.180153] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.187586] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.194695] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.202442] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:24 2020][8592449.209722] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.217594] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.224561] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:24 2020][8592449.230005] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.236482] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.244053] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:24 2020][8592449.249128] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592449.255397] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:24 2020][8592449.262011] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592449.268313] ll_ost_io03_010 D ffff9c7a37149040 0 68395 2 0x00000080 [Tue Mar 17 17:59:24 2020][8592449.275586] Call Trace: [Tue Mar 17 17:59:24 2020][8592449.278219] [] schedule+0x29/0x70 [Tue Mar 17 17:59:24 2020][8592449.283367] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 17:59:24 2020][8592449.289832] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:24 2020][8592449.295931] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 17:59:24 2020][8592449.301677] [] iget_locked+0x79/0x1a0 [Tue Mar 17 17:59:24 2020][8592449.307183] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.313636] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.320084] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.327500] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.334556] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.341524] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.348676] [] ? iova_rcache_get+0xba/0x140 [Tue Mar 17 17:59:24 2020][8592449.354704] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 17:59:24 2020][8592449.362020] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 17:59:24 2020][8592449.369105] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 17:59:24 2020][8592449.376623] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 17:59:24 2020][8592449.383691] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 17:59:24 2020][8592449.390322] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 17:59:24 2020][8592449.396703] [] ofd_preprw+0x496/0x11b0 [ofd] [Tue Mar 17 17:59:24 2020][8592449.402832] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.409927] [] tgt_brw_read+0x9db/0x1e50 [ptlrpc] [Tue Mar 17 17:59:24 2020][8592449.416459] [] ? kmalloc_order_trace+0x2e/0xa0 [Tue Mar 17 17:59:24 2020][8592449.422727] [] ? __kmalloc+0x211/0x230 [Tue Mar 17 17:59:25 2020][8592449.428339] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 17:59:25 2020][8592449.435680] [] ? null_alloc_rs+0x186/0x340 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.442411] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.449766] [] ? lustre_pack_reply_flags+0x6f/0x1e0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.457282] [] ? lustre_pack_reply+0x11/0x20 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.464195] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.471301] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.479047] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:25 2020][8592449.486315] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.494179] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.501155] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:25 2020][8592449.506589] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.513076] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.520651] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592449.525720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.532000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592449.538616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.544897] ll_ost_io01_012 D ffff9c7a3714d140 0 68396 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592449.552168] Call Trace: [Tue Mar 17 17:59:25 2020][8592449.554806] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.562467] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592449.567626] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.574679] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:25 2020][8592449.580688] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.587932] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.594646] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.602416] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:25 2020][8592449.608604] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.615402] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.622731] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.630309] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.637462] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:25 2020][8592449.643740] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:25 2020][8592449.650537] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:25 2020][8592449.656766] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.663472] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:25 2020][8592449.669026] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.676111] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.683857] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:25 2020][8592449.691125] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.698980] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.705945] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:25 2020][8592449.711390] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.717864] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.725438] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592449.730526] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.736800] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592449.743419] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.749702] ll_ost_io03_011 D ffff9c7a3714b0c0 0 68397 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592449.756973] Call Trace: [Tue Mar 17 17:59:25 2020][8592449.759609] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.767264] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592449.772430] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.779483] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:25 2020][8592449.785501] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.792753] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.799466] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.807223] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:25 2020][8592449.813407] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592449.820203] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.827535] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.835110] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.842252] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:25 2020][8592449.848540] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:25 2020][8592449.855337] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:25 2020][8592449.861554] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.868315] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.874959] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.882013] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:25 2020][8592449.888065] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.895492] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.902572] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.910334] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:25 2020][8592449.917591] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.925451] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.932444] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:25 2020][8592449.937880] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.944353] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592449.951938] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592449.956995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.963284] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592449.969900] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592449.976166] ll_ost_io03_012 D ffff9c8a37574100 0 68398 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592449.983459] Call Trace: [Tue Mar 17 17:59:25 2020][8592449.986097] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592449.993751] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592449.998930] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:25 2020][8592450.005980] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:25 2020][8592450.011996] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592450.019239] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:25 2020][8592450.025946] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592450.033701] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:25 2020][8592450.039887] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:25 2020][8592450.046687] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592450.054023] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:25 2020][8592450.061600] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:25 2020][8592450.068767] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:25 2020][8592450.075040] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:25 2020][8592450.081834] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:25 2020][8592450.088067] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.094771] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:25 2020][8592450.100298] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.107390] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.115141] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:25 2020][8592450.122408] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.130264] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.137258] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:25 2020][8592450.142691] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.149164] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.156752] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592450.161807] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.168097] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592450.174713] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.180987] ldlm_bl_03 S ffff9c8a37571040 0 68399 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592450.188281] Call Trace: [Tue Mar 17 17:59:25 2020][8592450.190912] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 17:59:25 2020][8592450.196785] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 17:59:25 2020][8592450.204208] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592450.209380] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.216451] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:25 2020][8592450.222247] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.229836] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592450.234893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.241175] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592450.247790] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.254064] ldlm_cn00_003 S ffff9c8a37570000 0 68444 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592450.261350] Call Trace: [Tue Mar 17 17:59:25 2020][8592450.263982] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:25 2020][8592450.270081] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592450.275273] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.282155] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:25 2020][8592450.288773] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:25 2020][8592450.294563] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.301033] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.308606] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592450.313692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.319966] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592450.326602] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.332871] ldlm_cn03_003 S ffff9c8a393a0000 0 68469 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592450.340138] Call Trace: [Tue Mar 17 17:59:25 2020][8592450.342767] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:25 2020][8592450.348876] [] schedule+0x29/0x70 [Tue Mar 17 17:59:25 2020][8592450.354050] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.360924] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:25 2020][8592450.366715] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.373207] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:25 2020][8592450.380781] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:25 2020][8592450.385855] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.392126] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:25 2020][8592450.398747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:25 2020][8592450.405033] ll_ost_create01 S ffff9c7ae992b0c0 0 68470 2 0x00000080 [Tue Mar 17 17:59:25 2020][8592450.412306] Call Trace: [Tue Mar 17 17:59:25 2020][8592450.414935] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:25 2020][8592450.421027] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.426211] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.433099] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592450.438880] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.445365] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.452939] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.458006] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.464274] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.470888] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.477169] ldlm_cn01_003 S ffff9c8a04e930c0 0 68471 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.484440] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.487069] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592450.493162] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.498367] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.505242] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592450.511016] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.517506] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.525073] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.530134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.536416] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.543029] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.549311] ll_ost01_013 S ffff9c8a2f2630c0 0 68472 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.556598] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.559227] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592450.565321] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.570514] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.577395] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592450.583177] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.589668] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.597240] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.602291] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.608574] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.615204] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.621472] ll_ost02_013 D ffff9c8a2f266180 0 68473 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.628758] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.631388] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.636539] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:26 2020][8592450.643602] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:26 2020][8592450.649617] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:26 2020][8592450.656857] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:26 2020][8592450.663566] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:26 2020][8592450.671062] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:26 2020][8592450.677260] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:26 2020][8592450.684053] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:26 2020][8592450.691382] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:26 2020][8592450.698960] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:26 2020][8592450.706100] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:26 2020][8592450.712388] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:26 2020][8592450.718491] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:26 2020][8592450.725224] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.733011] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.741016] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:26 2020][8592450.747524] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.754608] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.762356] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:26 2020][8592450.769621] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.777476] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.784442] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:26 2020][8592450.789885] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.796362] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.803935] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.809010] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.815292] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.821908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.828197] ll_ost02_014 S ffff9c8a3a225140 0 68474 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.835467] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.838098] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592450.844190] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.849373] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.856254] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592450.862029] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.868517] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.876096] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.881154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.887434] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.894049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.900315] ll_ost02_015 S ffff9c8a3a222080 0 68475 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.907600] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.910230] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592450.916324] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.921513] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.928387] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592450.934161] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.940666] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592450.948236] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592450.953296] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.959579] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592450.966202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592450.972473] ll_ost01_014 D ffff9c8a3a224100 0 68476 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592450.979761] Call Trace: [Tue Mar 17 17:59:26 2020][8592450.982389] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:26 2020][8592450.988396] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592450.995036] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:26 2020][8592451.001396] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:26 2020][8592451.006742] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.013752] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.020832] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.028578] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:26 2020][8592451.035851] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.043707] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.050682] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:26 2020][8592451.056111] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.062612] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.070184] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592451.075236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.081516] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592451.088134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.094408] ll_ost00_010 S ffff9c8a3a220000 0 68477 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592451.101703] Call Trace: [Tue Mar 17 17:59:26 2020][8592451.104330] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592451.110424] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592451.115614] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.122502] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592451.128280] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.134769] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.142342] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592451.147398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.153684] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592451.160302] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.166576] ll_ost02_016 S ffff9c8a3a221040 0 68478 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592451.173868] Call Trace: [Tue Mar 17 17:59:26 2020][8592451.176499] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592451.182600] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592451.187799] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.194672] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592451.200464] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.206955] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.214527] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592451.219596] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.225864] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592451.232476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.238757] ll_ost00_011 S ffff9c8a3a226180 0 68479 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592451.246045] Call Trace: [Tue Mar 17 17:59:26 2020][8592451.248675] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:26 2020][8592451.254768] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592451.259951] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.266833] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:26 2020][8592451.272614] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.279102] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:26 2020][8592451.286671] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:26 2020][8592451.291747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.298014] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:26 2020][8592451.304628] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.310929] ll_ost02_017 D ffff9c8a18e9b0c0 0 68480 2 0x00000080 [Tue Mar 17 17:59:26 2020][8592451.318206] Call Trace: [Tue Mar 17 17:59:26 2020][8592451.320835] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:26 2020][8592451.326153] [] schedule+0x29/0x70 [Tue Mar 17 17:59:26 2020][8592451.331309] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:26 2020][8592451.337329] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:26 2020][8592451.344991] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:26 2020][8592451.350327] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:26 2020][8592451.356164] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:26 2020][8592451.361482] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:26 2020][8592451.367688] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:26 2020][8592451.373095] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:26 2020][8592451.378504] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:26 2020][8592451.384097] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:26 2020][8592451.389413] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:26 2020][8592451.395855] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:26 2020][8592451.401977] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 17:59:26 2020][8592451.408861] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 17:59:26 2020][8592451.415653] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:26 2020][8592451.423259] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.431536] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.439384] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:27 2020][8592451.444706] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.451345] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 17:59:27 2020][8592451.458181] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.466132] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.473307] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.480042] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 17:59:27 2020][8592451.486844] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.493925] [] ? osd_oi_insert+0x4d0/0x520 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.501064] [] ? osd_destroy+0x1af/0x760 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.508032] [] ? osd_ref_del+0x1ad/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.515012] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:27 2020][8592451.521196] [] ofd_destroy+0x2c5/0x960 [ofd] [Tue Mar 17 17:59:27 2020][8592451.527296] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:27 2020][8592451.534043] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.541811] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.549825] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:27 2020][8592451.556321] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.563417] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.571170] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:27 2020][8592451.578435] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.586290] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.593272] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:27 2020][8592451.598701] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.605175] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.612763] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592451.617817] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.624098] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592451.630728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.636998] ll_ost01_015 S ffff9c8a18e9e180 0 68481 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592451.644290] Call Trace: [Tue Mar 17 17:59:27 2020][8592451.646922] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:27 2020][8592451.653021] [] schedule+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592451.658205] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.665086] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:27 2020][8592451.670867] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.677358] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.684933] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592451.689994] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.696298] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592451.702915] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.709182] ll_ost02_018 D ffff9c8a2e5ca080 0 68482 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592451.716466] Call Trace: [Tue Mar 17 17:59:27 2020][8592451.719103] [] ? jbd2_journal_dirty_metadata+0xd1/0x260 [jbd2] [Tue Mar 17 17:59:27 2020][8592451.726759] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592451.733394] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:27 2020][8592451.739748] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:27 2020][8592451.745095] [] tgt_last_rcvd_update+0x295/0xc90 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.752291] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.759018] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 17:59:27 2020][8592451.765816] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.772887] [] ? osd_oi_insert+0x4d0/0x520 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.780029] [] ? osd_destroy+0x1af/0x760 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.786997] [] ? osd_ref_del+0x1ad/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592451.793977] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:27 2020][8592451.800160] [] ofd_destroy+0x2c5/0x960 [ofd] [Tue Mar 17 17:59:27 2020][8592451.806260] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:27 2020][8592451.813004] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.820791] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.828796] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:27 2020][8592451.835286] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.842368] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.850117] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:27 2020][8592451.857383] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.865238] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.872218] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:27 2020][8592451.877649] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.884122] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.891722] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592451.896780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.903047] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592451.909676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592451.915943] ll_ost02_019 D ffff9c8a2e5cc100 0 68483 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592451.923212] Call Trace: [Tue Mar 17 17:59:27 2020][8592451.925855] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592451.932472] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:27 2020][8592451.938849] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:27 2020][8592451.944195] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.951190] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.958297] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.966044] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:27 2020][8592451.973294] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.981172] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592451.988141] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:27 2020][8592451.993574] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.000063] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.007639] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592452.012699] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.018993] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592452.025611] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.031879] ll_ost00_012 S ffff9c8a2e5c9040 0 68484 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592452.039173] Call Trace: [Tue Mar 17 17:59:27 2020][8592452.041801] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:27 2020][8592452.047895] [] schedule+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592452.053088] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.059967] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:27 2020][8592452.065743] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.072230] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.079811] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592452.084867] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.091148] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592452.097764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.104038] ll_ost_io02_013 D ffff9c8a2e5cb0c0 0 68485 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592452.111323] Call Trace: [Tue Mar 17 17:59:27 2020][8592452.113953] [] schedule+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592452.119110] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.126189] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.133322] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:27 2020][8592452.139363] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:27 2020][8592452.145546] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:27 2020][8592452.152340] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:27 2020][8592452.158573] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.165306] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.171954] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.179019] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:27 2020][8592452.184542] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.191620] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.199397] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:27 2020][8592452.206660] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.214525] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.221505] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:27 2020][8592452.226936] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.233423] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.240988] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:27 2020][8592452.246056] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.252327] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:27 2020][8592452.258946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:27 2020][8592452.265226] ll_ost_io03_013 D ffff9c8a2e5cd140 0 68486 2 0x00000080 [Tue Mar 17 17:59:27 2020][8592452.272516] Call Trace: [Tue Mar 17 17:59:27 2020][8592452.275155] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.282817] [] schedule+0x29/0x70 [Tue Mar 17 17:59:27 2020][8592452.287982] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:27 2020][8592452.295036] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:27 2020][8592452.301052] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:27 2020][8592452.308294] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:27 2020][8592452.315001] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.322760] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:27 2020][8592452.328952] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:27 2020][8592452.335749] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.343094] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.350673] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:27 2020][8592452.357862] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:27 2020][8592452.364139] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:27 2020][8592452.370947] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:27 2020][8592452.377168] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.383897] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.390561] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.397615] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:27 2020][8592452.403159] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.410238] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:27 2020][8592452.418001] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:27 2020][8592452.425256] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.433130] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.440089] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:28 2020][8592452.445520] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.452008] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.459582] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:28 2020][8592452.464645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.470945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:28 2020][8592452.477565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.483841] ll_ost02_020 D ffff9c8a32084100 0 68487 2 0x00000080 [Tue Mar 17 17:59:28 2020][8592452.491134] Call Trace: [Tue Mar 17 17:59:28 2020][8592452.493764] [] schedule+0x29/0x70 [Tue Mar 17 17:59:28 2020][8592452.498916] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.505980] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:28 2020][8592452.511992] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.519234] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.525945] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.533459] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:28 2020][8592452.539647] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.546444] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.553777] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.561352] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.568491] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:28 2020][8592452.574779] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:28 2020][8592452.580877] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:28 2020][8592452.587608] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.595398] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.603425] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:28 2020][8592452.609903] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.616998] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.624749] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:28 2020][8592452.631999] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.639876] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.646844] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:28 2020][8592452.652281] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.658770] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.666343] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:28 2020][8592452.671398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.677700] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:28 2020][8592452.684318] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.690584] ll_ost_io00_014 D ffff9c8a32086180 0 68488 2 0x00000080 [Tue Mar 17 17:59:28 2020][8592452.697870] Call Trace: [Tue Mar 17 17:59:28 2020][8592452.700502] [] schedule+0x29/0x70 [Tue Mar 17 17:59:28 2020][8592452.705656] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.712723] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.719860] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:28 2020][8592452.725869] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:28 2020][8592452.732088] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:28 2020][8592452.738880] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:28 2020][8592452.745096] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.751843] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.758490] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.765557] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:28 2020][8592452.771078] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.778173] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.785925] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:28 2020][8592452.793175] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.801064] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.808028] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:28 2020][8592452.813463] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.819968] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.827536] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:28 2020][8592452.832597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.838878] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:28 2020][8592452.845493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592452.851759] ll_ost_io01_013 D ffff9c8a32085140 0 68489 2 0x00000080 [Tue Mar 17 17:59:28 2020][8592452.859047] Call Trace: [Tue Mar 17 17:59:28 2020][8592452.861674] [] schedule+0x29/0x70 [Tue Mar 17 17:59:28 2020][8592452.866818] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.873883] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:28 2020][8592452.879896] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.887138] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:28 2020][8592452.892981] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.899710] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.907020] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:28 2020][8592452.913225] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:28 2020][8592452.920025] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.927342] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.934928] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592452.942072] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:28 2020][8592452.948865] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:28 2020][8592452.955096] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.961833] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.968476] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592452.975567] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:28 2020][8592452.982882] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:28 2020][8592452.988649] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:28 2020][8592452.994184] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.001266] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.009029] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:28 2020][8592453.016283] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.024158] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.031126] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:28 2020][8592453.036554] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.043042] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.050611] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:28 2020][8592453.055669] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592453.061966] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:28 2020][8592453.068584] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592453.074860] ll_ost_io02_014 D ffff9c8a34ab1040 0 68490 2 0x00000080 [Tue Mar 17 17:59:28 2020][8592453.082152] Call Trace: [Tue Mar 17 17:59:28 2020][8592453.084784] [] schedule+0x29/0x70 [Tue Mar 17 17:59:28 2020][8592453.089936] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.097007] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.104142] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:28 2020][8592453.110152] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:28 2020][8592453.116354] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:28 2020][8592453.123161] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:28 2020][8592453.129377] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.136124] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.142775] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.149850] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:28 2020][8592453.157173] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:28 2020][8592453.162920] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:28 2020][8592453.169473] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.176921] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.184005] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.191753] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:28 2020][8592453.199049] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.206906] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.213872] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:28 2020][8592453.219328] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.225801] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.233375] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:28 2020][8592453.238449] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592453.244718] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:28 2020][8592453.251341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:28 2020][8592453.257642] ll_ost_io02_015 D ffff9c8a217f1040 0 68491 2 0x00000080 [Tue Mar 17 17:59:28 2020][8592453.264917] Call Trace: [Tue Mar 17 17:59:28 2020][8592453.267547] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:28 2020][8592453.274166] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:28 2020][8592453.280535] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:28 2020][8592453.285854] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:28 2020][8592453.291355] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.298357] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.305758] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:28 2020][8592453.312055] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.318953] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.327311] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.334555] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.342210] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:28 2020][8592453.347797] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.355491] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.363416] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.370816] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.377887] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.386070] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:28 2020][8592453.393066] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.400299] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:28 2020][8592453.407627] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:28 2020][8592453.414424] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:28 2020][8592453.420644] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:28 2020][8592453.427397] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.434047] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.441115] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:29 2020][8592453.448442] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:29 2020][8592453.454194] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:29 2020][8592453.460781] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.468212] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.475294] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.483053] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:29 2020][8592453.490311] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.498187] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.505153] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:29 2020][8592453.510591] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.517079] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.524655] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:29 2020][8592453.529741] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.536016] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:29 2020][8592453.542638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.548924] ll_ost_io00_015 D ffff9c8a217f2080 0 68492 2 0x00000080 [Tue Mar 17 17:59:29 2020][8592453.556198] Call Trace: [Tue Mar 17 17:59:29 2020][8592453.558827] [] schedule+0x29/0x70 [Tue Mar 17 17:59:29 2020][8592453.563977] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.571045] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:29 2020][8592453.577055] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.584296] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.591004] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.598774] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:29 2020][8592453.604961] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.611758] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.619091] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.626666] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.633804] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:29 2020][8592453.640095] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 17:59:29 2020][8592453.646632] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 17:59:29 2020][8592453.652936] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.660045] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.667799] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:29 2020][8592453.675057] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.682932] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.689894] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:29 2020][8592453.695347] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.701823] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.709394] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:29 2020][8592453.714470] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.720739] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:29 2020][8592453.727358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.733652] ll_ost_io00_016 D ffff9c8a217f0000 0 68493 2 0x00000080 [Tue Mar 17 17:59:29 2020][8592453.740928] Call Trace: [Tue Mar 17 17:59:29 2020][8592453.743556] [] schedule+0x29/0x70 [Tue Mar 17 17:59:29 2020][8592453.748701] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.755766] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:29 2020][8592453.761777] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.769017] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 17:59:29 2020][8592453.774942] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.781673] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.788995] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:29 2020][8592453.795179] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592453.801991] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.809310] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.816896] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.824039] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:29 2020][8592453.830855] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:29 2020][8592453.837069] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.843772] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:29 2020][8592453.849313] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.856420] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.864168] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:29 2020][8592453.871436] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.879300] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.886278] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:29 2020][8592453.891709] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.898197] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.905763] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:29 2020][8592453.910821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.917110] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:29 2020][8592453.923731] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592453.930032] ll_ost_io00_017 D ffff9c8a30bb30c0 0 68494 2 0x00000080 [Tue Mar 17 17:59:29 2020][8592453.937307] Call Trace: [Tue Mar 17 17:59:29 2020][8592453.939937] [] schedule+0x29/0x70 [Tue Mar 17 17:59:29 2020][8592453.945092] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592453.952154] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:29 2020][8592453.958166] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:29 2020][8592453.964372] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:29 2020][8592453.971167] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:29 2020][8592453.977384] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.984143] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.990789] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592453.997841] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:29 2020][8592454.003895] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.011322] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.018403] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.026164] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:29 2020][8592454.033418] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.041295] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.048261] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:29 2020][8592454.053692] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.060194] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.067762] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:29 2020][8592454.072822] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592454.079104] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:29 2020][8592454.085720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592454.091997] ll_ost_io02_016 D ffff9c8a314d0000 0 68495 2 0x00000080 [Tue Mar 17 17:59:29 2020][8592454.099288] Call Trace: [Tue Mar 17 17:59:29 2020][8592454.101926] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.109589] [] schedule+0x29/0x70 [Tue Mar 17 17:59:29 2020][8592454.114766] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.121815] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:29 2020][8592454.127825] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.135066] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.141772] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.149530] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:29 2020][8592454.155713] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.162512] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.169845] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.177419] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.184560] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:29 2020][8592454.190867] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:29 2020][8592454.197662] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:29 2020][8592454.203894] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.210643] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.217293] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.224360] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:29 2020][8592454.229878] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.236958] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.244721] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:29 2020][8592454.251990] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.259878] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.266847] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:29 2020][8592454.272296] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.278763] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.286338] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:29 2020][8592454.291413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592454.297682] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:29 2020][8592454.304304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:29 2020][8592454.310592] ll_ost_io01_014 D ffff9c8a314d4100 0 68496 2 0x00000080 [Tue Mar 17 17:59:29 2020][8592454.317882] Call Trace: [Tue Mar 17 17:59:29 2020][8592454.320511] [] schedule+0x29/0x70 [Tue Mar 17 17:59:29 2020][8592454.325663] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.332727] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:29 2020][8592454.338737] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.345981] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:29 2020][8592454.351815] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.358565] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.365872] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:29 2020][8592454.372078] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:29 2020][8592454.378891] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.386210] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.393799] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:29 2020][8592454.400942] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:29 2020][8592454.407736] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:29 2020][8592454.413968] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.420700] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:29 2020][8592454.427348] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.434435] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:30 2020][8592454.441748] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:30 2020][8592454.447495] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:30 2020][8592454.453049] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.460129] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.467876] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:30 2020][8592454.475142] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.483005] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.489970] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:30 2020][8592454.495412] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.501882] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.509467] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:30 2020][8592454.514522] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.520818] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:30 2020][8592454.527437] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.533703] ll_ost_io03_014 D ffff9c8a314d6180 0 68497 2 0x00000080 [Tue Mar 17 17:59:30 2020][8592454.540996] Call Trace: [Tue Mar 17 17:59:30 2020][8592454.543625] [] schedule+0x29/0x70 [Tue Mar 17 17:59:30 2020][8592454.548767] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.555839] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:30 2020][8592454.561856] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.569095] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:30 2020][8592454.574933] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.581660] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.588972] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:30 2020][8592454.595176] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.601979] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.609301] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.616888] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.624032] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:30 2020][8592454.630825] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:30 2020][8592454.637057] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.643777] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:30 2020][8592454.649298] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.656389] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.664140] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:30 2020][8592454.671409] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.679272] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.686237] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:30 2020][8592454.691678] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.698154] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.705728] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:30 2020][8592454.710809] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.717083] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:30 2020][8592454.723704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.729991] ll_ost_io03_015 D ffff9c8a314d2080 0 68498 2 0x00000080 [Tue Mar 17 17:59:30 2020][8592454.737264] Call Trace: [Tue Mar 17 17:59:30 2020][8592454.739895] [] schedule+0x29/0x70 [Tue Mar 17 17:59:30 2020][8592454.745044] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.752112] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:30 2020][8592454.758123] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.765362] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:30 2020][8592454.771198] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.777942] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.785255] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:30 2020][8592454.791445] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.798256] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.805577] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.813164] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592454.820306] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:30 2020][8592454.827101] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:30 2020][8592454.833332] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.840079] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.846722] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.853808] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:30 2020][8592454.861124] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:30 2020][8592454.866877] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:30 2020][8592454.873442] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.880876] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.887961] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.895722] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:30 2020][8592454.902977] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.910840] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.917828] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:30 2020][8592454.923257] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.929732] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592454.937317] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:30 2020][8592454.942374] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.948662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:30 2020][8592454.955279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592454.961553] ll_ost_io03_016 D ffff9c8a314d1040 0 68499 2 0x00000080 [Tue Mar 17 17:59:30 2020][8592454.968849] Call Trace: [Tue Mar 17 17:59:30 2020][8592454.971476] [] schedule+0x29/0x70 [Tue Mar 17 17:59:30 2020][8592454.976617] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.983697] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:30 2020][8592454.989714] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592454.996953] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:30 2020][8592455.002793] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.009519] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.016830] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:30 2020][8592455.023032] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.029828] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.037142] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.044732] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.051875] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:30 2020][8592455.058666] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:30 2020][8592455.064898] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.071632] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.078281] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.085344] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:30 2020][8592455.091381] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.098814] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.105923] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.113667] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:30 2020][8592455.120920] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.128796] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.135763] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:30 2020][8592455.141201] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.147689] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.155264] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:30 2020][8592455.160338] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592455.166609] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:30 2020][8592455.173230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592455.179531] ll_ost02_021 D ffff9c8a314d30c0 0 68500 2 0x00000080 [Tue Mar 17 17:59:30 2020][8592455.186807] Call Trace: [Tue Mar 17 17:59:30 2020][8592455.189435] [] schedule+0x29/0x70 [Tue Mar 17 17:59:30 2020][8592455.194581] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.201645] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:30 2020][8592455.207670] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.214908] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.221640] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.229136] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:30 2020][8592455.235319] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.242147] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.249459] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.257034] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:30 2020][8592455.264188] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:30 2020][8592455.270467] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:30 2020][8592455.276580] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:30 2020][8592455.283307] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.291094] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.299098] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:30 2020][8592455.305577] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.312684] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.320428] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:30 2020][8592455.327683] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.335557] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.342554] [] ? ptlrpc_server_post_idle_rqbds+0x7e/0xf0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.350501] [] ? ptlrpc_grow_req_bufs+0x50/0x2a0 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.357768] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.364238] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:30 2020][8592455.371826] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:30 2020][8592455.376879] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592455.383174] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:30 2020][8592455.389794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:30 2020][8592455.396070] ll_ost_io02_017 D ffff9c8a2bfe0000 0 68501 2 0x00000080 [Tue Mar 17 17:59:30 2020][8592455.403363] Call Trace: [Tue Mar 17 17:59:30 2020][8592455.405993] [] schedule+0x29/0x70 [Tue Mar 17 17:59:30 2020][8592455.411144] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.418209] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:30 2020][8592455.424220] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:30 2020][8592455.431460] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:31 2020][8592455.437298] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.444025] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.451336] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:31 2020][8592455.457526] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.464338] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.471658] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.479246] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.486388] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:31 2020][8592455.493181] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:31 2020][8592455.499413] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.506164] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.512814] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.519896] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:31 2020][8592455.527205] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:31 2020][8592455.532972] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:31 2020][8592455.538495] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.545588] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.553340] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:31 2020][8592455.560607] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.568468] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.575435] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:31 2020][8592455.580896] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.587370] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.594945] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:31 2020][8592455.600018] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592455.606290] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:31 2020][8592455.612908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592455.619189] ll_ost_io01_015 D ffff9c8a2bfe5140 0 68502 2 0x00000080 [Tue Mar 17 17:59:31 2020][8592455.626460] Call Trace: [Tue Mar 17 17:59:31 2020][8592455.629092] [] schedule+0x29/0x70 [Tue Mar 17 17:59:31 2020][8592455.634243] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.641322] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:31 2020][8592455.647337] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.654580] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.661294] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.669049] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:31 2020][8592455.675236] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.682045] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.689366] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.696955] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.704112] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:31 2020][8592455.710389] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:31 2020][8592455.717195] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:31 2020][8592455.723408] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.730137] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.737584] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.744665] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.752426] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:31 2020][8592455.759675] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.767534] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.774530] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:31 2020][8592455.779964] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.786437] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.794026] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:31 2020][8592455.799078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592455.805360] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:31 2020][8592455.811975] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592455.818241] ll_ost_io02_018 D ffff9c8a2bfe30c0 0 68503 2 0x00000080 [Tue Mar 17 17:59:31 2020][8592455.825527] Call Trace: [Tue Mar 17 17:59:31 2020][8592455.828155] [] schedule+0x29/0x70 [Tue Mar 17 17:59:31 2020][8592455.833299] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.840377] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:31 2020][8592455.846393] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.853636] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.860340] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.868098] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:31 2020][8592455.874284] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592455.881081] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.888413] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.895989] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592455.903127] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:31 2020][8592455.909429] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 17:59:31 2020][8592455.915963] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 17:59:31 2020][8592455.922269] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.929370] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.937120] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:31 2020][8592455.944387] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.952249] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.959215] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:31 2020][8592455.965028] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.971498] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592455.979071] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:31 2020][8592455.984145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592455.990414] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:31 2020][8592455.997027] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592456.003306] ll_ost_io03_017 D ffff9c8a2bfee180 0 68504 2 0x00000080 [Tue Mar 17 17:59:31 2020][8592456.010580] Call Trace: [Tue Mar 17 17:59:31 2020][8592456.013208] [] schedule+0x29/0x70 [Tue Mar 17 17:59:31 2020][8592456.018354] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.025422] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.032561] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:31 2020][8592456.038607] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:31 2020][8592456.044794] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:31 2020][8592456.051589] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:31 2020][8592456.057817] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.064553] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.071216] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.078288] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:31 2020][8592456.085617] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:31 2020][8592456.091365] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:31 2020][8592456.097926] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.105388] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.112464] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.120213] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:31 2020][8592456.127479] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.135335] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.142301] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:31 2020][8592456.147750] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.154219] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.161789] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:31 2020][8592456.166871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592456.173146] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:31 2020][8592456.179767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:31 2020][8592456.186054] ll_ost_io03_018 D ffff9c8a2bfe8000 0 68505 2 0x00000080 [Tue Mar 17 17:59:31 2020][8592456.193326] Call Trace: [Tue Mar 17 17:59:31 2020][8592456.195958] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:31 2020][8592456.201276] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:31 2020][8592456.207917] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:31 2020][8592456.214278] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:31 2020][8592456.219600] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:31 2020][8592456.225128] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.232102] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.239497] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:31 2020][8592456.245793] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.252678] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.261031] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.268274] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.275932] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:31 2020][8592456.281520] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.289195] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.297135] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.304538] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.311611] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.319789] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:31 2020][8592456.326776] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.334003] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:31 2020][8592456.341332] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:31 2020][8592456.348128] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:31 2020][8592456.354347] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.361092] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.367741] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.374823] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:31 2020][8592456.380862] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.388293] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.395386] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:31 2020][8592456.403136] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:31 2020][8592456.410405] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.418267] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.425235] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:32 2020][8592456.430685] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.437176] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.444753] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:32 2020][8592456.449826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.456096] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:32 2020][8592456.462732] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.469001] ll_ost_io02_019 D ffff9c8a2bfec100 0 68506 2 0x00000080 [Tue Mar 17 17:59:32 2020][8592456.476278] Call Trace: [Tue Mar 17 17:59:32 2020][8592456.478920] [] schedule+0x29/0x70 [Tue Mar 17 17:59:32 2020][8592456.484068] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.491137] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.498291] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:32 2020][8592456.504303] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:32 2020][8592456.510507] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:32 2020][8592456.517302] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:32 2020][8592456.523520] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.530237] [] ? load_balance+0x1be/0x9a0 [Tue Mar 17 17:59:32 2020][8592456.536072] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:32 2020][8592456.541591] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.548684] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.556434] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:32 2020][8592456.563686] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.571576] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.578539] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:32 2020][8592456.583977] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.590464] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.598030] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:32 2020][8592456.603082] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.609364] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:32 2020][8592456.615979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.622246] ll_ost_io00_018 D ffff9c8a2bfe9040 0 68507 2 0x00000080 [Tue Mar 17 17:59:32 2020][8592456.629557] Call Trace: [Tue Mar 17 17:59:32 2020][8592456.632186] [] schedule+0x29/0x70 [Tue Mar 17 17:59:32 2020][8592456.637341] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.644411] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.651548] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:32 2020][8592456.657565] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:32 2020][8592456.663769] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:32 2020][8592456.670566] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:32 2020][8592456.676782] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.683528] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.690172] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.697266] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:32 2020][8592456.704579] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:32 2020][8592456.710335] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:32 2020][8592456.716910] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.724339] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.731431] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.739181] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:32 2020][8592456.746435] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.754308] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.761285] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:32 2020][8592456.766715] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.773203] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.780776] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:32 2020][8592456.785847] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.792123] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:32 2020][8592456.798742] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592456.805024] ll_ost_io01_016 D ffff9c8a2bfeb0c0 0 68508 2 0x00000080 [Tue Mar 17 17:59:32 2020][8592456.812295] Call Trace: [Tue Mar 17 17:59:32 2020][8592456.814925] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:32 2020][8592456.821546] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:32 2020][8592456.827934] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:32 2020][8592456.833255] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:32 2020][8592456.838758] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.845747] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.853146] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:32 2020][8592456.859180] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:32 2020][8592456.865546] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:32 2020][8592456.872173] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:32 2020][8592456.877662] [] ? dqget+0x1a7/0x450 [Tue Mar 17 17:59:32 2020][8592456.882890] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:32 2020][8592456.890057] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:32 2020][8592456.895290] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:32 2020][8592456.900956] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.908371] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:32 2020][8592456.915344] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.922657] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.929553] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592456.936868] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:32 2020][8592456.943749] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:32 2020][8592456.949994] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.956725] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.963376] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.970457] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:32 2020][8592456.977764] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:32 2020][8592456.983517] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:32 2020][8592456.989051] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592456.996133] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.003880] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:32 2020][8592457.011146] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.019001] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.025995] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:32 2020][8592457.031431] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.037903] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.045490] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:32 2020][8592457.050548] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592457.056836] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:32 2020][8592457.063457] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592457.069739] ll_ost_io01_017 D ffff9c8a2bfed140 0 68509 2 0x00000080 [Tue Mar 17 17:59:32 2020][8592457.077021] Call Trace: [Tue Mar 17 17:59:32 2020][8592457.079658] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.087333] [] schedule+0x29/0x70 [Tue Mar 17 17:59:32 2020][8592457.092493] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.099541] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:32 2020][8592457.105572] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.112803] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.119526] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.127272] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:32 2020][8592457.133469] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.140262] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.147577] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.155178] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.162316] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:32 2020][8592457.168591] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:32 2020][8592457.175401] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:32 2020][8592457.181619] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.188323] [] ? ___slab_alloc+0x24e/0x4f0 [Tue Mar 17 17:59:32 2020][8592457.194294] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.200944] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.208011] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:32 2020][8592457.214074] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.221518] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.228616] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.236364] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:32 2020][8592457.243631] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.251487] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.258467] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:32 2020][8592457.263897] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.270385] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:32 2020][8592457.277953] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:32 2020][8592457.283013] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592457.289308] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:32 2020][8592457.295926] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:32 2020][8592457.302193] ll_ost_io02_020 D ffff9c8a36f49040 0 68510 2 0x00000080 [Tue Mar 17 17:59:32 2020][8592457.309478] Call Trace: [Tue Mar 17 17:59:32 2020][8592457.312116] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.319779] [] schedule+0x29/0x70 [Tue Mar 17 17:59:32 2020][8592457.324945] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.331999] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:32 2020][8592457.338016] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.345272] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.351981] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.359760] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:32 2020][8592457.365961] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:32 2020][8592457.372756] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.380086] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.387661] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:32 2020][8592457.394800] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:32 2020][8592457.401090] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:32 2020][8592457.407888] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:32 2020][8592457.414106] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.420865] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.427517] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.434582] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:33 2020][8592457.440105] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.447199] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.454949] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:33 2020][8592457.462201] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.470077] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.477044] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:33 2020][8592457.482472] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.488978] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.496546] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.501607] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.507896] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592457.514510] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.520777] ll_ost_create01 S ffff9c8a36f4d140 0 68519 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592457.528072] Call Trace: [Tue Mar 17 17:59:33 2020][8592457.530702] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592457.536802] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592457.542007] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.548885] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592457.554668] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.561154] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.568721] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.573773] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.580056] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592457.586670] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.592944] ll_ost_create02 S ffff9c8a36f48000 0 68532 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592457.600230] Call Trace: [Tue Mar 17 17:59:33 2020][8592457.602861] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592457.608959] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592457.614155] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.621035] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592457.626816] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.633306] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.640880] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.645946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.652216] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592457.658837] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.665138] ll_ost03_013 S ffff9c8a334c6180 0 68533 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592457.672405] Call Trace: [Tue Mar 17 17:59:33 2020][8592457.675035] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592457.681129] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592457.686321] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.693205] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592457.698985] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.705473] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.713038] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.718092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.724386] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592457.731004] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.737270] ll_ost03_014 S ffff9c5981784100 0 68534 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592457.744557] Call Trace: [Tue Mar 17 17:59:33 2020][8592457.747186] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592457.753280] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592457.758471] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.765353] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592457.771150] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.777628] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.785200] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.790289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.796560] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592457.803174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592457.809462] ll_ost03_015 D ffff9c59817830c0 0 68535 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592457.816734] Call Trace: [Tue Mar 17 17:59:33 2020][8592457.819364] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592457.824516] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:33 2020][8592457.831578] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:33 2020][8592457.837591] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:33 2020][8592457.844854] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:33 2020][8592457.851571] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592457.859058] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:33 2020][8592457.865264] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:33 2020][8592457.872062] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592457.879378] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:33 2020][8592457.886969] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592457.894109] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:33 2020][8592457.900385] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:33 2020][8592457.906499] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:33 2020][8592457.913247] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.921015] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.929042] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:33 2020][8592457.935523] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.942613] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.950364] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:33 2020][8592457.957631] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.965488] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.972452] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:33 2020][8592457.977902] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.984386] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592457.991962] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592457.997035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.003306] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592458.009925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.016207] ll_ost_create03 S ffff9c5981782080 0 68536 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592458.023478] Call Trace: [Tue Mar 17 17:59:33 2020][8592458.026109] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592458.032210] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592458.037400] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.044297] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592458.050073] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.056564] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.064138] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592458.069198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.075480] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592458.082096] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.088368] ll_ost01_016 D ffff9c5981786180 0 68537 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592458.095655] Call Trace: [Tue Mar 17 17:59:33 2020][8592458.098286] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592458.103434] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.110516] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:33 2020][8592458.116531] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.123773] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.130480] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592458.137975] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:33 2020][8592458.144159] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.150958] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592458.158287] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:33 2020][8592458.165864] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592458.173004] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:33 2020][8592458.179307] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:33 2020][8592458.185406] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:33 2020][8592458.192138] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.199928] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.207955] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:33 2020][8592458.214436] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.221534] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.229285] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:33 2020][8592458.236567] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.244427] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.251422] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:33 2020][8592458.256857] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.263334] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.270922] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592458.275977] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.282259] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592458.288874] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.295165] ll_ost01_017 S ffff9c5981785140 0 68538 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592458.302434] Call Trace: [Tue Mar 17 17:59:33 2020][8592458.305066] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:33 2020][8592458.311165] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592458.316363] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.323238] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:33 2020][8592458.329022] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.335510] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:33 2020][8592458.343079] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:33 2020][8592458.348139] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.354445] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:33 2020][8592458.361061] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:33 2020][8592458.367348] ll_ost03_016 D ffff9c7a1f3c30c0 0 68539 2 0x00000080 [Tue Mar 17 17:59:33 2020][8592458.374638] Call Trace: [Tue Mar 17 17:59:33 2020][8592458.377267] [] schedule+0x29/0x70 [Tue Mar 17 17:59:33 2020][8592458.382419] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.389482] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:33 2020][8592458.395495] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.402736] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:33 2020][8592458.409449] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:33 2020][8592458.416935] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:34 2020][8592458.423138] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:34 2020][8592458.429947] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.437264] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.444856] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.451995] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:34 2020][8592458.458269] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:34 2020][8592458.464384] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:34 2020][8592458.471111] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.478900] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.486904] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:34 2020][8592458.493381] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.500489] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.508233] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:34 2020][8592458.515485] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.523362] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.530330] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:34 2020][8592458.535767] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.542257] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.549829] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592458.554883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.561172] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592458.567786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.574080] ll_ost01_018 D ffff9c7ae981b0c0 0 68540 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592458.581356] Call Trace: [Tue Mar 17 17:59:34 2020][8592458.583984] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592458.589129] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:34 2020][8592458.596193] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:34 2020][8592458.602206] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:34 2020][8592458.609448] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:34 2020][8592458.616156] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.623660] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:34 2020][8592458.629864] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:34 2020][8592458.636660] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.643989] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.651565] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592458.658704] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:34 2020][8592458.664993] [] ofd_attr_set+0x464/0xb60 [ofd] [Tue Mar 17 17:59:34 2020][8592458.671186] [] ofd_setattr_hdl+0x31d/0x8e0 [ofd] [Tue Mar 17 17:59:34 2020][8592458.677662] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.684755] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.692505] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:34 2020][8592458.699785] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.707644] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.714611] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:34 2020][8592458.720063] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.726551] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.734119] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592458.739194] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.745464] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592458.752098] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.758367] ll_ost03_017 S ffff9c8a3a00d140 0 68541 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592458.765635] Call Trace: [Tue Mar 17 17:59:34 2020][8592458.768292] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:34 2020][8592458.774393] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592458.779577] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.786457] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:34 2020][8592458.792232] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.798722] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.806295] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592458.811356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.817637] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592458.824267] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.830536] ll_ost00_013 D ffff9c8a3a00c100 0 68542 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592458.837821] Call Trace: [Tue Mar 17 17:59:34 2020][8592458.840452] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592458.847072] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:34 2020][8592458.853447] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:34 2020][8592458.858799] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.865798] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.872892] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.880641] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:34 2020][8592458.887924] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.895781] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.902747] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:34 2020][8592458.908199] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.914676] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592458.922247] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592458.927315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.933584] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592458.940205] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592458.946482] ll_ost03_018 D ffff9c8a3a008000 0 68543 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592458.953756] Call Trace: [Tue Mar 17 17:59:34 2020][8592458.956413] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:34 2020][8592458.961733] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592458.966896] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:34 2020][8592458.972904] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:34 2020][8592458.978216] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:34 2020][8592458.984411] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:34 2020][8592458.989814] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:34 2020][8592458.995222] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:34 2020][8592459.001249] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:34 2020][8592459.006579] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:34 2020][8592459.013456] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.019571] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:34 2020][8592459.025153] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:34 2020][8592459.032036] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.040314] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:34 2020][8592459.047888] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.056169] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.063141] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.070903] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.077801] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.084942] [] evict+0xb4/0x180 [Tue Mar 17 17:59:34 2020][8592459.089927] [] iput+0xfc/0x190 [Tue Mar 17 17:59:34 2020][8592459.094816] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:34 2020][8592459.102152] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:34 2020][8592459.109577] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:34 2020][8592459.116730] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:34 2020][8592459.123353] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:34 2020][8592459.130102] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.137879] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.145888] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:34 2020][8592459.152393] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.159479] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.167224] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:34 2020][8592459.174494] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.182356] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.189321] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:34 2020][8592459.194773] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.201250] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.208837] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592459.213921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.220193] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592459.226826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.233097] ll_ost03_019 S ffff9c8a3a31b0c0 0 68544 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592459.240383] Call Trace: [Tue Mar 17 17:59:34 2020][8592459.243011] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:34 2020][8592459.249104] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592459.254296] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.261178] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:34 2020][8592459.266962] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.273462] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.281034] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592459.286092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.292372] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592459.298988] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.305257] ll_ost03_020 S ffff9c8a3a319040 0 68545 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592459.312550] Call Trace: [Tue Mar 17 17:59:34 2020][8592459.315178] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:34 2020][8592459.321271] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592459.326455] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.333354] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:34 2020][8592459.339136] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.345627] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.353200] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:34 2020][8592459.358297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.364572] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:34 2020][8592459.371211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:34 2020][8592459.377500] ll_ost00_014 S ffff9c8a3a318000 0 68546 2 0x00000080 [Tue Mar 17 17:59:34 2020][8592459.384770] Call Trace: [Tue Mar 17 17:59:34 2020][8592459.387414] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:34 2020][8592459.393509] [] schedule+0x29/0x70 [Tue Mar 17 17:59:34 2020][8592459.398705] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.405582] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:34 2020][8592459.411366] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:34 2020][8592459.417854] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.425429] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592459.430496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.436764] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592459.443392] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.449660] ll_ost00_015 S ffff9c8a163cc100 0 68547 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592459.456929] Call Trace: [Tue Mar 17 17:59:35 2020][8592459.459572] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:35 2020][8592459.465683] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592459.470870] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.477748] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:35 2020][8592459.483538] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.490007] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.497595] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592459.502649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.508929] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592459.515560] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.521829] ll_ost03_021 D ffff9c8a163cd140 0 68548 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592459.529122] Call Trace: [Tue Mar 17 17:59:35 2020][8592459.531752] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592459.536901] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 17:59:35 2020][8592459.543362] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.549459] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 17:59:35 2020][8592459.555212] [] iget_locked+0x79/0x1a0 [Tue Mar 17 17:59:35 2020][8592459.560721] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.567171] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.573621] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.581050] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.588110] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.595079] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.602233] [] ? kiblnd_queue_tx+0x36/0x50 [ko2iblnd] [Tue Mar 17 17:59:35 2020][8592459.609130] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 17:59:35 2020][8592459.616466] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.623532] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 17:59:35 2020][8592459.631039] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 17:59:35 2020][8592459.638138] [] ? null_alloc_rs+0x16d/0x340 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.644864] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 17:59:35 2020][8592459.651514] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 17:59:35 2020][8592459.657874] [] ofd_destroy_by_fid+0xa2/0x4a0 [ofd] [Tue Mar 17 17:59:35 2020][8592459.664522] [] ? lustre_msg_buf_v2+0x1e0/0x1e0 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.671616] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.678269] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.685334] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:35 2020][8592459.691807] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.698900] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.706653] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:35 2020][8592459.713932] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.721791] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.728758] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:35 2020][8592459.734206] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.740677] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.748262] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592459.753316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.759598] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592459.766213] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.772477] ll_ost02_022 D ffff9c8a32e7b0c0 0 68549 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592459.779775] Call Trace: [Tue Mar 17 17:59:35 2020][8592459.782401] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592459.787546] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:35 2020][8592459.794612] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:35 2020][8592459.800623] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:35 2020][8592459.807870] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:35 2020][8592459.814584] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.822073] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:35 2020][8592459.828275] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:35 2020][8592459.835066] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.842383] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.849986] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592459.857123] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:35 2020][8592459.863397] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:35 2020][8592459.869513] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:35 2020][8592459.876246] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.884022] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.892047] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:35 2020][8592459.898529] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.905628] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.913378] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:35 2020][8592459.920660] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.928518] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.935483] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:35 2020][8592459.940935] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.947412] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592459.954982] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592459.960051] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.966321] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592459.972940] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592459.979234] ll_ost01_019 D ffff9c8a32e7c100 0 68550 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592459.986511] Call Trace: [Tue Mar 17 17:59:35 2020][8592459.989137] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592459.994281] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:35 2020][8592460.001348] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:35 2020][8592460.007362] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:35 2020][8592460.014603] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:35 2020][8592460.021317] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592460.028813] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:35 2020][8592460.034997] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:35 2020][8592460.041794] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592460.049160] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:35 2020][8592460.056736] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:35 2020][8592460.063877] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:35 2020][8592460.070167] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:35 2020][8592460.076268] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:35 2020][8592460.083001] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.090789] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.098795] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:35 2020][8592460.105273] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.112378] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.120125] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:35 2020][8592460.127376] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.135252] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.142220] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:35 2020][8592460.147655] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.154144] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.161713] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592460.166787] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.173057] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592460.179678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.185979] ll_ost03_022 S ffff9c8a32e7d140 0 68551 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592460.193254] Call Trace: [Tue Mar 17 17:59:35 2020][8592460.195884] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:35 2020][8592460.201984] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592460.207168] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.214064] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:35 2020][8592460.219841] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.226330] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.233906] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592460.238996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.245266] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592460.251881] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.258169] ll_ost03_023 S ffff9c8a32e7a080 0 68552 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592460.265440] Call Trace: [Tue Mar 17 17:59:35 2020][8592460.268083] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:35 2020][8592460.274179] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592460.279372] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.286252] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:35 2020][8592460.292035] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.298536] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.306107] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592460.311159] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.317439] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592460.324055] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.330321] ll_ost02_023 S ffff9c8a32e78000 0 68553 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592460.337607] Call Trace: [Tue Mar 17 17:59:35 2020][8592460.340236] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:35 2020][8592460.346328] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592460.351513] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.358407] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:35 2020][8592460.364186] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.370674] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:35 2020][8592460.378250] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:35 2020][8592460.383311] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.389600] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:35 2020][8592460.396214] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:35 2020][8592460.402494] ll_ost00_016 S ffff9c8a32e79040 0 68554 2 0x00000080 [Tue Mar 17 17:59:35 2020][8592460.409767] Call Trace: [Tue Mar 17 17:59:35 2020][8592460.412395] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:35 2020][8592460.418487] [] schedule+0x29/0x70 [Tue Mar 17 17:59:35 2020][8592460.423684] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.430562] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592460.436345] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.442833] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.450398] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.455453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.461740] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.468357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.474644] ll_ost02_024 D ffff9c8a36925140 0 68555 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.481935] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.484565] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.489715] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:36 2020][8592460.496779] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:36 2020][8592460.502792] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592460.510035] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:36 2020][8592460.516746] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592460.524246] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:36 2020][8592460.530429] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592460.537243] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592460.544575] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:36 2020][8592460.552153] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592460.559305] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:36 2020][8592460.565584] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:36 2020][8592460.571685] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:36 2020][8592460.578432] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.586198] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.594210] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:36 2020][8592460.600702] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.607784] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.615531] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:36 2020][8592460.622812] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.630670] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.637637] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:36 2020][8592460.643088] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.649564] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.657134] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.662201] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.668473] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.675092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.681386] ll_ost00_017 S ffff9c8a23ef1040 0 68556 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.688662] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.691290] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592460.697384] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.702574] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.709449] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592460.715231] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.721725] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.729295] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.734356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.740661] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.747277] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.753544] ll_ost02_025 S ffff9c8a23ef4100 0 68557 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.760829] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.763459] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592460.769552] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.774745] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.781626] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592460.787408] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.793899] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.801470] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.806552] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.812823] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.819438] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.825726] ll_ost01_020 S ffff9c8a36a58000 0 68558 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.832996] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.835627] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592460.841728] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.846911] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.853792] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592460.859575] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.866064] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.873629] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.878683] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.884973] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.891587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.897859] ll_ost02_026 S ffff9c8a36a5a080 0 68559 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.905148] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.907777] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592460.913871] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.919075] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.925951] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592460.931734] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.938225] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.945798] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592460.950870] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.957143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592460.963765] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592460.970067] ll_ost00_018 S ffff9c8a33e3b0c0 0 68560 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592460.977341] Call Trace: [Tue Mar 17 17:59:36 2020][8592460.979972] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592460.986073] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592460.991263] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592460.998137] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592461.003918] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.010407] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.017975] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592461.023035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.029317] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592461.035932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.042199] ll_ost02_027 D ffff9c8a33e3e180 0 68561 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592461.049500] Call Trace: [Tue Mar 17 17:59:36 2020][8592461.052130] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592461.057276] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.064339] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:36 2020][8592461.070349] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.077591] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.084300] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.091780] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:36 2020][8592461.097978] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.104793] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.112112] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.119699] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.126840] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:36 2020][8592461.133118] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:36 2020][8592461.139231] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:36 2020][8592461.145965] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.153744] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.161750] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:36 2020][8592461.168228] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.175338] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.183089] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:36 2020][8592461.190341] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.198215] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.205177] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:36 2020][8592461.210611] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.217114] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.224683] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592461.229767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.236040] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592461.242660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.248962] ll_ost00_019 S ffff9c8a33e38000 0 68562 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592461.256238] Call Trace: [Tue Mar 17 17:59:36 2020][8592461.258868] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:36 2020][8592461.264967] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592461.270151] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.277033] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:36 2020][8592461.282829] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.289297] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:36 2020][8592461.296869] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:36 2020][8592461.301951] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.308226] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:36 2020][8592461.314844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:36 2020][8592461.321123] ll_ost03_024 D ffff9c8a33e39040 0 68563 2 0x00000080 [Tue Mar 17 17:59:36 2020][8592461.328395] Call Trace: [Tue Mar 17 17:59:36 2020][8592461.331027] [] schedule+0x29/0x70 [Tue Mar 17 17:59:36 2020][8592461.336176] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.343244] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:36 2020][8592461.349255] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.356513] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.363224] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.370726] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:36 2020][8592461.376910] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:36 2020][8592461.383708] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.391037] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.398614] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:36 2020][8592461.405755] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:36 2020][8592461.412043] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:36 2020][8592461.418146] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:36 2020][8592461.424881] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.432680] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.440689] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:37 2020][8592461.447168] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.454261] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.462012] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:37 2020][8592461.469263] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.477146] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.484106] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:37 2020][8592461.489541] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.496030] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.503611] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592461.508668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.514956] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592461.521573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.527845] ll_ost01_021 S ffff9c8a33e3a080 0 68564 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592461.535132] Call Trace: [Tue Mar 17 17:59:37 2020][8592461.537760] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:37 2020][8592461.543856] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592461.549046] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.555928] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:37 2020][8592461.561711] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.568214] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.575782] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592461.580856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.587125] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592461.593752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.600024] ll_ost00_020 S ffff9c8a33e3d140 0 68565 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592461.607317] Call Trace: [Tue Mar 17 17:59:37 2020][8592461.609947] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:37 2020][8592461.616049] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592461.621254] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.628132] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:37 2020][8592461.634751] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:37 2020][8592461.640550] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.647026] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.654597] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592461.659663] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.665934] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592461.672555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.678844] ll_ost03_025 D ffff9c8a1e316180 0 68566 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592461.686132] Call Trace: [Tue Mar 17 17:59:37 2020][8592461.688763] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592461.693912] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:37 2020][8592461.700981] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:37 2020][8592461.706989] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:37 2020][8592461.714233] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:37 2020][8592461.720941] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:37 2020][8592461.728430] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:37 2020][8592461.734636] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:37 2020][8592461.741434] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:37 2020][8592461.748752] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592461.756352] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:37 2020][8592461.763491] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:37 2020][8592461.769765] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:37 2020][8592461.775880] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:37 2020][8592461.782616] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.790388] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.798415] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:37 2020][8592461.804896] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.811988] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.819739] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:37 2020][8592461.827017] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.834884] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.841853] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:37 2020][8592461.847302] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.853770] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.861342] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592461.866409] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.872677] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592461.879291] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.885586] ll_ost01_022 S ffff9c8a1e314100 0 68567 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592461.892860] Call Trace: [Tue Mar 17 17:59:37 2020][8592461.895491] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:37 2020][8592461.901591] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592461.906783] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.913664] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:37 2020][8592461.919448] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.925934] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592461.933500] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592461.938568] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.944851] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592461.951474] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592461.957757] ll_ost01_023 D ffff9c8a1e3130c0 0 68568 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592461.965027] Call Trace: [Tue Mar 17 17:59:37 2020][8592461.967658] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:37 2020][8592461.972992] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592461.978135] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:37 2020][8592461.984143] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:37 2020][8592461.990171] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:37 2020][8592461.996005] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:37 2020][8592462.001318] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:37 2020][8592462.007525] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:37 2020][8592462.012931] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:37 2020][8592462.018331] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:37 2020][8592462.024351] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:37 2020][8592462.029668] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:37 2020][8592462.036549] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.042664] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:37 2020][8592462.048246] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:37 2020][8592462.055122] [] ? __getblk+0x2d/0x300 [Tue Mar 17 17:59:37 2020][8592462.060561] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:37 2020][8592462.068139] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.076413] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.084085] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:37 2020][8592462.091052] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.098192] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.105696] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.112666] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:37 2020][8592462.119459] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 17:59:37 2020][8592462.125574] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:37 2020][8592462.132323] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.140090] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.148107] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:37 2020][8592462.154579] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.161659] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.169420] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:37 2020][8592462.176675] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.184535] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.191516] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:37 2020][8592462.196945] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.203420] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.211021] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592462.216096] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.222364] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592462.229001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.235282] ll_ost03_026 S ffff9c8a1e312080 0 68569 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592462.242572] Call Trace: [Tue Mar 17 17:59:37 2020][8592462.245200] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:37 2020][8592462.251292] [] schedule+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592462.256475] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.263362] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:37 2020][8592462.269141] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.275628] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.283195] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592462.288248] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.294528] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592462.301143] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.307409] ll_ost01_024 D ffff9c8a1e315140 0 68570 2 0x00000080 [Tue Mar 17 17:59:37 2020][8592462.314696] Call Trace: [Tue Mar 17 17:59:37 2020][8592462.317326] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:37 2020][8592462.323342] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:37 2020][8592462.329989] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:37 2020][8592462.336348] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:37 2020][8592462.341694] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.348704] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.355784] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.363553] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:37 2020][8592462.370824] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.378680] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.385660] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:37 2020][8592462.391105] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.397581] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:37 2020][8592462.405167] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:37 2020][8592462.410225] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:37 2020][8592462.416514] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:37 2020][8592462.423127] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.429395] ll_ost02_028 D ffff9c8a1e311040 0 68571 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592462.436688] Call Trace: [Tue Mar 17 17:59:38 2020][8592462.439317] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:38 2020][8592462.445324] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592462.451964] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:38 2020][8592462.458324] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:38 2020][8592462.463677] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.470687] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.477768] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.485516] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:38 2020][8592462.492782] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.500638] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.507617] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:38 2020][8592462.513047] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.519523] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.527122] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592462.532182] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.538457] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592462.545092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.551362] ll_ost03_027 D ffff9c7ae984b0c0 0 68572 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592462.558640] Call Trace: [Tue Mar 17 17:59:38 2020][8592462.561279] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:38 2020][8592462.566597] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592462.571751] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:38 2020][8592462.577776] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:38 2020][8592462.583090] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:38 2020][8592462.589283] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:38 2020][8592462.594687] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:38 2020][8592462.600095] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:38 2020][8592462.606122] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:38 2020][8592462.611439] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:38 2020][8592462.618313] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.624427] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:38 2020][8592462.630008] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.636919] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.645181] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.652775] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.661046] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.668028] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.675778] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.682673] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.689815] [] evict+0xb4/0x180 [Tue Mar 17 17:59:38 2020][8592462.694801] [] iput+0xfc/0x190 [Tue Mar 17 17:59:38 2020][8592462.699691] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.707054] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:38 2020][8592462.714459] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:38 2020][8592462.721611] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:38 2020][8592462.728252] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:38 2020][8592462.734985] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.742755] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.750776] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:38 2020][8592462.757257] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.764344] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.772104] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:38 2020][8592462.779374] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.787237] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.794217] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:38 2020][8592462.799648] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.806122] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.813710] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592462.818764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.825045] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592462.831662] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.837934] ll_ost01_025 S ffff9c8a2ee2b0c0 0 68573 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592462.845238] Call Trace: [Tue Mar 17 17:59:38 2020][8592462.847866] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:38 2020][8592462.853959] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592462.859140] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.866015] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:38 2020][8592462.871791] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.878279] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592462.885853] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592462.890928] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.897199] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592462.903820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592462.910122] ll_ost02_029 D ffff9c8a2ee29040 0 68574 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592462.917396] Call Trace: [Tue Mar 17 17:59:38 2020][8592462.920027] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592462.925178] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.932242] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:38 2020][8592462.938255] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.945498] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.952216] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.959717] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:38 2020][8592462.965915] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:38 2020][8592462.972707] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.980040] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.987615] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592462.994756] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:38 2020][8592463.001044] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:38 2020][8592463.007149] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:38 2020][8592463.013882] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.021666] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.029676] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:38 2020][8592463.036150] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.043259] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.051013] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:38 2020][8592463.058263] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.066140] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.073106] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:38 2020][8592463.078537] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.085025] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.092598] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592463.097653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.103956] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592463.110572] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.116838] ll_ost01_026 S ffff9c8a2ee28000 0 68575 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592463.124124] Call Trace: [Tue Mar 17 17:59:38 2020][8592463.126753] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:38 2020][8592463.132857] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592463.138050] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.144928] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:38 2020][8592463.150702] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.157191] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.164773] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592463.169827] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.176109] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592463.182724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.188999] ll_ost03_028 D ffff9c8a2ee2c100 0 68576 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592463.196292] Call Trace: [Tue Mar 17 17:59:38 2020][8592463.198922] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592463.204074] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:38 2020][8592463.211141] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:38 2020][8592463.217167] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:38 2020][8592463.224423] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:38 2020][8592463.231149] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592463.238648] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:38 2020][8592463.244832] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:38 2020][8592463.251643] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592463.258965] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:38 2020][8592463.266559] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:38 2020][8592463.273701] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:38 2020][8592463.279977] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:38 2020][8592463.286091] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:38 2020][8592463.292831] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.300599] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.308617] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:38 2020][8592463.315098] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.322176] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.329939] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:38 2020][8592463.337195] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.345069] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.352029] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:38 2020][8592463.357809] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.364333] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.371901] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:38 2020][8592463.376962] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.383244] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:38 2020][8592463.389862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:38 2020][8592463.396149] ll_ost00_021 S ffff9c8a2ee2e180 0 68577 2 0x00000080 [Tue Mar 17 17:59:38 2020][8592463.403418] Call Trace: [Tue Mar 17 17:59:38 2020][8592463.406049] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:38 2020][8592463.412150] [] schedule+0x29/0x70 [Tue Mar 17 17:59:38 2020][8592463.417340] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:38 2020][8592463.424228] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:38 2020][8592463.430004] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.436493] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.444061] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592463.449122] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.455411] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592463.462025] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.468291] ll_ost00_022 S ffff9c8a2ee2a080 0 68578 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592463.475577] Call Trace: [Tue Mar 17 17:59:39 2020][8592463.478206] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592463.484300] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592463.489503] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.496382] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:39 2020][8592463.503003] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592463.508800] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.515270] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.522841] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592463.527918] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.534186] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592463.540806] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.547102] ll_ost02_030 D ffff9c597feb8000 0 68579 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592463.554374] Call Trace: [Tue Mar 17 17:59:39 2020][8592463.557006] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592463.562157] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.569221] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:39 2020][8592463.575232] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.582475] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.589186] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.596688] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:39 2020][8592463.602872] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.609669] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.617016] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.624595] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.631734] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:39 2020][8592463.638020] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:39 2020][8592463.644116] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:39 2020][8592463.650849] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.658638] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.666643] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:39 2020][8592463.673121] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.680226] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.687971] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:39 2020][8592463.695225] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.703099] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.710059] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:39 2020][8592463.715488] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.721974] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.729542] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592463.734593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.740880] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592463.747514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.753782] ll_ost01_027 D ffff9c597feb9040 0 68580 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592463.761069] Call Trace: [Tue Mar 17 17:59:39 2020][8592463.763698] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592463.768839] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.775905] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:39 2020][8592463.781919] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.789161] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.795879] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.803382] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:39 2020][8592463.809577] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:39 2020][8592463.816372] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.823701] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.831278] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:39 2020][8592463.838432] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:39 2020][8592463.844711] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:39 2020][8592463.850812] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:39 2020][8592463.857555] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.865323] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.873327] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:39 2020][8592463.879832] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.886918] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.894664] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:39 2020][8592463.901932] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.909787] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.916753] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:39 2020][8592463.922207] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.928680] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.936252] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592463.941320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.947603] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592463.954218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592463.960506] ll_ost01_028 S ffff9c597feba080 0 68581 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592463.967779] Call Trace: [Tue Mar 17 17:59:39 2020][8592463.970408] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592463.976510] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592463.981701] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592463.988584] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592463.994365] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.000853] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.008433] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.013489] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.019771] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592464.026386] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.032651] ll_ost03_029 S ffff9c597febb0c0 0 68582 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592464.039937] Call Trace: [Tue Mar 17 17:59:39 2020][8592464.042566] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592464.048660] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592464.053851] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.060732] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592464.066508] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.073010] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.080580] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.085641] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.091932] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592464.098545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.104818] ll_ost00_023 S ffff9c597febc100 0 68583 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592464.112107] Call Trace: [Tue Mar 17 17:59:39 2020][8592464.114734] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592464.120827] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592464.126023] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.132901] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592464.138683] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.145171] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.152738] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.157791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.164080] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592464.170696] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.176969] ll_ost01_029 D ffff9c597febd140 0 68584 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592464.184257] Call Trace: [Tue Mar 17 17:59:39 2020][8592464.186886] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:39 2020][8592464.192900] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592464.199547] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:39 2020][8592464.205900] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:39 2020][8592464.211248] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.218277] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.225361] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.233124] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:39 2020][8592464.240381] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.248240] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.255235] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:39 2020][8592464.260667] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.267142] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.274730] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.279784] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.286073] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592464.292690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.298962] ll_ost00_024 S ffff9c597febe180 0 68585 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592464.306249] Call Trace: [Tue Mar 17 17:59:39 2020][8592464.308878] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592464.314970] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592464.320167] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.327044] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592464.332826] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.339316] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.346881] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.351935] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.358223] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:39 2020][8592464.364862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.371132] ll_ost00_025 S ffff9c597ffa8000 0 68586 2 0x00000080 [Tue Mar 17 17:59:39 2020][8592464.378425] Call Trace: [Tue Mar 17 17:59:39 2020][8592464.381054] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:39 2020][8592464.387148] [] schedule+0x29/0x70 [Tue Mar 17 17:59:39 2020][8592464.392340] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.399220] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:39 2020][8592464.405004] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.411493] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:39 2020][8592464.419067] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:39 2020][8592464.424127] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:39 2020][8592464.430408] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.437041] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.443307] ll_ost01_030 S ffff9c597ffa9040 0 68587 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.450592] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.453222] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592464.459323] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.464504] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.471378] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592464.477154] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.483644] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.491216] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.496297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.502571] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.509193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.515482] ll_ost03_030 S ffff9c597ffaa080 0 68588 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.522752] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.525382] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592464.531483] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.536672] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.543547] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592464.549330] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.555834] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.563403] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.568465] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.574752] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.581367] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.587650] ll_ost03_031 S ffff9c597ffab0c0 0 68589 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.594921] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.597550] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592464.603651] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.608842] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.615729] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592464.621508] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.627997] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.635571] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.640631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.646912] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.653527] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.659794] ll_ost02_031 S ffff9c597ffac100 0 68590 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.667078] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.669709] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592464.675809] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.681007] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.687883] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592464.693667] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.700158] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.707729] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.712780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.719064] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.725677] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.731945] ll_ost00_026 S ffff9c597ffad140 0 68591 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.739248] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.741877] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592464.747980] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.753168] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.760047] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592464.765825] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.772313] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.779879] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.784932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.791214] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592464.797843] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.804111] ll_ost01_031 D ffff9c597ffae180 0 68592 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592464.811398] Call Trace: [Tue Mar 17 17:59:40 2020][8592464.814028] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592464.819183] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:40 2020][8592464.826246] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:40 2020][8592464.832257] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592464.839499] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:40 2020][8592464.846218] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592464.853717] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:40 2020][8592464.859916] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592464.866707] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592464.874043] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:40 2020][8592464.881616] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592464.888755] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:40 2020][8592464.895046] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:40 2020][8592464.901147] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:40 2020][8592464.907881] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.915667] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.923674] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:40 2020][8592464.930151] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.937259] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.945012] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:40 2020][8592464.952278] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.960134] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.967113] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:40 2020][8592464.972543] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.979032] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592464.986600] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592464.991659] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592464.997952] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592465.004565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592465.010831] ll_ost03_032 S ffff9c597fa10000 0 68593 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592465.018116] Call Trace: [Tue Mar 17 17:59:40 2020][8592465.020746] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:40 2020][8592465.026847] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592465.032039] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.038919] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:40 2020][8592465.044696] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.051184] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.058773] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592465.063827] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592465.070109] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592465.076724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592465.082991] ll_ost02_032 D ffff9c597fa11040 0 68594 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592465.090286] Call Trace: [Tue Mar 17 17:59:40 2020][8592465.092912] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592465.098056] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.105120] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:40 2020][8592465.111134] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.118388] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.125103] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.132589] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:40 2020][8592465.138785] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.145578] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.152895] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.160483] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.167625] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:40 2020][8592465.173899] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:40 2020][8592465.180014] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:40 2020][8592465.186765] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.194530] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.202549] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:40 2020][8592465.209031] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.216119] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.223898] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:40 2020][8592465.231151] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.239028] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.245993] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:40 2020][8592465.251421] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.257927] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.265494] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:40 2020][8592465.270555] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592465.276837] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:40 2020][8592465.283453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:40 2020][8592465.289727] ll_ost02_033 D ffff9c597fa12080 0 68595 2 0x00000080 [Tue Mar 17 17:59:40 2020][8592465.297020] Call Trace: [Tue Mar 17 17:59:40 2020][8592465.299649] [] schedule+0x29/0x70 [Tue Mar 17 17:59:40 2020][8592465.304794] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.311858] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:40 2020][8592465.317873] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.325126] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.331838] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.339340] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:40 2020][8592465.345527] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:40 2020][8592465.352322] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.359654] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.367231] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:40 2020][8592465.374385] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:40 2020][8592465.380661] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:40 2020][8592465.386762] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:40 2020][8592465.393523] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.401296] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.409306] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:40 2020][8592465.415798] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.422879] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:40 2020][8592465.430625] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:41 2020][8592465.437897] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.445755] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.452736] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:41 2020][8592465.458182] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.464660] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.472244] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.477300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.483590] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.490203] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.496471] ll_ost00_027 S ffff9c597fa130c0 0 68596 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.503757] Call Trace: [Tue Mar 17 17:59:41 2020][8592465.506386] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592465.512488] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592465.517692] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.524569] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592465.530351] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.536841] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.544415] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.549469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.555758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.562373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.568648] ll_ost03_033 S ffff9c597fa14100 0 68597 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.575942] Call Trace: [Tue Mar 17 17:59:41 2020][8592465.578570] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592465.584664] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592465.589861] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.596743] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592465.602534] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.609004] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.616593] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.621653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.627933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.634548] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.640815] ll_ost02_034 D ffff9c597fa15140 0 68598 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.648101] Call Trace: [Tue Mar 17 17:59:41 2020][8592465.650730] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592465.655874] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:41 2020][8592465.662938] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:41 2020][8592465.668950] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:41 2020][8592465.676191] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:41 2020][8592465.682900] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:41 2020][8592465.690379] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:41 2020][8592465.696577] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:41 2020][8592465.703383] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:41 2020][8592465.710705] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592465.718299] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:41 2020][8592465.725441] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:41 2020][8592465.731716] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:41 2020][8592465.737831] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:41 2020][8592465.744567] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.752339] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.760357] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:41 2020][8592465.766836] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.773916] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.781690] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:41 2020][8592465.788941] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.796804] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.803783] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:41 2020][8592465.809211] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.815685] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.823273] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.828330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.834605] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.841252] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.847524] ll_ost02_035 S ffff9c597fa16180 0 68599 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.854794] Call Trace: [Tue Mar 17 17:59:41 2020][8592465.857438] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592465.863532] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592465.868716] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.875598] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592465.881381] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.887872] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.895442] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.900524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.906798] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.913432] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.919702] ll_ost00_028 S ffff9c5980158000 0 68600 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.926969] Call Trace: [Tue Mar 17 17:59:41 2020][8592465.929611] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592465.935708] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592465.940894] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.947772] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592465.953562] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.960029] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592465.967631] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592465.972691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.978965] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592465.985600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592465.991873] ll_ost01_032 S ffff9c5980159040 0 68601 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592465.999147] Call Trace: [Tue Mar 17 17:59:41 2020][8592466.001776] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592466.007888] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592466.013063] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.019941] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592466.025728] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.032200] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.039772] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592466.044848] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.051116] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592466.057738] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.064026] ll_ost03_034 S ffff9c598015a080 0 68602 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592466.071297] Call Trace: [Tue Mar 17 17:59:41 2020][8592466.073928] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592466.080028] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592466.085234] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.092109] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592466.097894] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.104379] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.111946] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592466.116998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.123281] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592466.129895] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.136162] ll_ost00_029 S ffff9c598015b0c0 0 68603 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592466.143456] Call Trace: [Tue Mar 17 17:59:41 2020][8592466.146084] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:41 2020][8592466.152178] [] schedule+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592466.157359] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.164234] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:41 2020][8592466.170017] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.176506] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.184073] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:41 2020][8592466.189133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.195414] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:41 2020][8592466.202042] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:41 2020][8592466.208312] ll_ost02_036 D ffff9c598015c100 0 68604 2 0x00000080 [Tue Mar 17 17:59:41 2020][8592466.215597] Call Trace: [Tue Mar 17 17:59:41 2020][8592466.218228] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:41 2020][8592466.224864] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:41 2020][8592466.231239] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:41 2020][8592466.236559] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:41 2020][8592466.242060] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.249044] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.256455] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:41 2020][8592466.262377] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:41 2020][8592466.268682] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:41 2020][8592466.274005] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.280973] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.288475] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.295790] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.303452] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.310603] [] evict+0xb4/0x180 [Tue Mar 17 17:59:41 2020][8592466.315571] [] iput+0xfc/0x190 [Tue Mar 17 17:59:41 2020][8592466.320459] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:41 2020][8592466.327826] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:41 2020][8592466.335228] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:41 2020][8592466.342390] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:41 2020][8592466.349029] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:41 2020][8592466.355762] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.363534] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.371559] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:41 2020][8592466.378043] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.385128] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.392894] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:41 2020][8592466.400163] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:41 2020][8592466.408023] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.415002] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:42 2020][8592466.420435] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.426921] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.434489] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592466.439564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.445833] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592466.452444] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.458726] ll_ost01_033 S ffff9c598015d140 0 68605 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592466.466016] Call Trace: [Tue Mar 17 17:59:42 2020][8592466.468645] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592466.474746] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592466.479936] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.486810] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592466.492591] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.499081] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.506649] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592466.511709] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.518000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592466.524628] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.530899] ll_ost03_035 S ffff9c598015e180 0 68606 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592466.538191] Call Trace: [Tue Mar 17 17:59:42 2020][8592466.540820] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592466.546913] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592466.552095] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.558968] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592466.564743] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.571232] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.578806] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592466.583861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.590165] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592466.596781] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.603048] ll_ost03_036 S ffff9c597f418000 0 68607 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592466.610341] Call Trace: [Tue Mar 17 17:59:42 2020][8592466.612970] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592466.619064] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592466.624255] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.631136] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592466.636910] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.643412] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.650983] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592466.656035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.662316] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592466.668933] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.675207] ll_ost01_034 D ffff9c597f419040 0 68608 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592466.682501] Call Trace: [Tue Mar 17 17:59:42 2020][8592466.685129] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:42 2020][8592466.690441] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592466.695596] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:42 2020][8592466.701650] [] ? at_measured+0x108/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.708187] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:42 2020][8592466.715427] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:42 2020][8592466.721180] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:42 2020][8592466.727016] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:42 2020][8592466.732344] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:42 2020][8592466.738526] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:42 2020][8592466.743933] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:42 2020][8592466.749348] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:42 2020][8592466.755355] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:42 2020][8592466.760668] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:42 2020][8592466.767569] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.773671] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:42 2020][8592466.779258] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:42 2020][8592466.786147] [] ? __getblk+0x2d/0x300 [Tue Mar 17 17:59:42 2020][8592466.791552] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:42 2020][8592466.799147] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.807414] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.815087] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:42 2020][8592466.822053] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.829223] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.836709] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.843677] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:42 2020][8592466.850484] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 17:59:42 2020][8592466.856586] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:42 2020][8592466.863320] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.871106] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.879111] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:42 2020][8592466.885590] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.892683] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.900435] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:42 2020][8592466.907716] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.915573] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.922540] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592466.928336] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.934814] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.942399] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592466.947453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.953735] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592466.960349] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592466.966615] ll_ost03_037 S ffff9c597f41a080 0 68609 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592466.973911] Call Trace: [Tue Mar 17 17:59:42 2020][8592466.976541] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592466.982642] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592466.987831] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592466.994706] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592467.000489] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.006977] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.014545] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592467.019605] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.025906] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592467.032528] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.038802] ll_ost01_035 S ffff9c597f41b0c0 0 68610 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592467.046087] Call Trace: [Tue Mar 17 17:59:42 2020][8592467.048716] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592467.054810] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592467.060001] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.066882] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592467.072673] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.079148] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.086719] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592467.091800] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.098074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592467.104694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.110973] ll_ost02_037 S ffff9c597f41c100 0 68611 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592467.118247] Call Trace: [Tue Mar 17 17:59:42 2020][8592467.120877] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:42 2020][8592467.126975] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592467.132161] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.139043] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:42 2020][8592467.144824] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.151312] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.158878] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592467.163931] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.170213] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592467.176830] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.183105] ll_ost01_036 D ffff9c597f41d140 0 68612 2 0x00000080 [Tue Mar 17 17:59:42 2020][8592467.190397] Call Trace: [Tue Mar 17 17:59:42 2020][8592467.193028] [] schedule+0x29/0x70 [Tue Mar 17 17:59:42 2020][8592467.198179] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:42 2020][8592467.205256] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:42 2020][8592467.211273] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:42 2020][8592467.218514] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:42 2020][8592467.225239] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:42 2020][8592467.232734] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:42 2020][8592467.238918] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:42 2020][8592467.245746] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:42 2020][8592467.253068] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:42 2020][8592467.260657] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:42 2020][8592467.267800] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:42 2020][8592467.274072] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:42 2020][8592467.280200] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:42 2020][8592467.286929] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.294703] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.302721] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:42 2020][8592467.309195] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.316283] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.324042] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:42 2020][8592467.331299] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.339174] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.344138] LNetError: 13721:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.202@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 17:59:42 2020][8592467.344141] LNetError: 13721:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 375 previous similar messages [Tue Mar 17 17:59:42 2020][8592467.367487] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:42 2020][8592467.372917] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.379390] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:42 2020][8592467.386975] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:42 2020][8592467.392030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.398312] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:42 2020][8592467.404926] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:42 2020][8592467.411193] ll_ost02_038 S ffff9c597f41e180 0 68613 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.418488] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.421118] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592467.427219] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.432408] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.439284] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592467.445068] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.451553] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.459120] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592467.464187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.470457] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592467.477078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.483371] ll_ost03_038 S ffff9c597f5d8000 0 68614 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.490647] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.493276] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592467.499368] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.504552] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.511432] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592467.517206] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.523697] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.531270] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592467.536325] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.542627] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592467.549247] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.555519] ll_ost01_037 D ffff9c597f5d9040 0 68615 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.562807] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.565435] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.570579] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:43 2020][8592467.577645] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:43 2020][8592467.583654] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:43 2020][8592467.590898] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:43 2020][8592467.597608] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:43 2020][8592467.605109] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:43 2020][8592467.611294] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:43 2020][8592467.618090] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:43 2020][8592467.625422] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:43 2020][8592467.632999] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:43 2020][8592467.640137] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:43 2020][8592467.646428] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:43 2020][8592467.652532] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:43 2020][8592467.659263] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.667065] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.675074] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:43 2020][8592467.681550] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.688646] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.696395] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:43 2020][8592467.703647] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.711522] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.718491] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:43 2020][8592467.723930] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.730416] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.737989] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592467.743070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.749343] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592467.755956] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.762236] ll_ost00_030 S ffff9c597f5da080 0 68616 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.769509] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.772137] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592467.778238] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.783421] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.790295] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:43 2020][8592467.796914] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592467.802705] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.809181] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.816753] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592467.821821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.828089] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592467.834704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.840993] ll_ost01_038 D ffff9c597f5db0c0 0 68617 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.848263] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.850891] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:43 2020][8592467.856905] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.863548] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:43 2020][8592467.869908] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:43 2020][8592467.875261] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.882269] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.889351] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.897101] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:43 2020][8592467.904365] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.912228] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.919208] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:43 2020][8592467.924654] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.931123] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.938709] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592467.943763] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.950044] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592467.956661] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592467.962936] ll_ost03_039 S ffff9c597f5dc100 0 68618 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592467.970232] Call Trace: [Tue Mar 17 17:59:43 2020][8592467.972859] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592467.978960] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592467.984164] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592467.991042] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592467.996824] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.003312] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.010881] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592468.015942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.022229] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592468.028845] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.035110] ll_ost01_039 S ffff9c597f5dd140 0 68619 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592468.042396] Call Trace: [Tue Mar 17 17:59:43 2020][8592468.045027] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592468.051128] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592468.056331] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.063209] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592468.068984] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.075472] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.083039] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592468.088114] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.094382] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592468.100995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.107289] ll_ost02_039 S ffff9c597f5de180 0 68620 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592468.114565] Call Trace: [Tue Mar 17 17:59:43 2020][8592468.117195] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592468.123296] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592468.128487] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.135369] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:43 2020][8592468.141151] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.147639] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.155207] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592468.160282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.166565] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592468.173182] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.179469] ll_ost00_031 D ffff9c597f268000 0 68621 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592468.186740] Call Trace: [Tue Mar 17 17:59:43 2020][8592468.189370] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592468.194513] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:43 2020][8592468.201578] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:43 2020][8592468.207591] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:43 2020][8592468.214836] [] ? kiblnd_send+0x357/0xa20 [ko2iblnd] [Tue Mar 17 17:59:43 2020][8592468.221544] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:43 2020][8592468.228316] [] ? lu_obj_hop_keycmp+0x1d/0x30 [obdclass] [Tue Mar 17 17:59:43 2020][8592468.235373] [] ? cfs_hash_bd_lookup_intent+0x63/0x170 [libcfs] [Tue Mar 17 17:59:43 2020][8592468.243064] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:43 2020][8592468.249252] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:43 2020][8592468.256051] [] ? ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 17:59:43 2020][8592468.263380] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:43 2020][8592468.270956] [] ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 17:59:43 2020][8592468.278093] [] dqget+0x3fa/0x450 [Tue Mar 17 17:59:43 2020][8592468.283168] [] dquot_get_dqblk+0x14/0x1f0 [Tue Mar 17 17:59:43 2020][8592468.289015] [] osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:43 2020][8592468.296681] [] lquotactl_slv+0x27d/0x9d0 [lquota] [Tue Mar 17 17:59:43 2020][8592468.303240] [] ofd_quotactl+0x13c/0x380 [ofd] [Tue Mar 17 17:59:43 2020][8592468.309456] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.316542] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.324304] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:43 2020][8592468.331560] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.339419] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.346400] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:43 2020][8592468.351831] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.358322] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.365923] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:43 2020][8592468.370983] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.377272] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:43 2020][8592468.383886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:43 2020][8592468.390153] ll_ost03_040 S ffff9c597f269040 0 68622 2 0x00000080 [Tue Mar 17 17:59:43 2020][8592468.397446] Call Trace: [Tue Mar 17 17:59:43 2020][8592468.400077] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:43 2020][8592468.406177] [] schedule+0x29/0x70 [Tue Mar 17 17:59:43 2020][8592468.411359] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:43 2020][8592468.418232] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592468.424008] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.430508] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.438081] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592468.443142] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.449430] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592468.456046] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.462314] ll_ost00_032 S ffff9c597f26a080 0 68623 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592468.469606] Call Trace: [Tue Mar 17 17:59:44 2020][8592468.472236] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:44 2020][8592468.478337] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592468.483542] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.490418] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592468.496204] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.502689] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.510255] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592468.515325] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.521591] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592468.528203] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.534486] ll_ost00_033 D ffff9c597f26b0c0 0 68624 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592468.541757] Call Trace: [Tue Mar 17 17:59:44 2020][8592468.544386] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592468.551007] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:44 2020][8592468.557390] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:44 2020][8592468.562742] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.569735] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.576828] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.584577] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:44 2020][8592468.591831] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.599706] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.606673] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:44 2020][8592468.612100] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.618604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.626174] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592468.631236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.637524] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592468.644141] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.650415] ll_ost03_041 S ffff9c597f26c100 0 68625 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592468.657725] Call Trace: [Tue Mar 17 17:59:44 2020][8592468.660356] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:44 2020][8592468.666455] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592468.671652] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.678529] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592468.684303] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.690793] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.698367] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592468.703434] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.709703] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592468.716316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.722597] ll_ost01_040 D ffff9c597f26d140 0 68626 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592468.729868] Call Trace: [Tue Mar 17 17:59:44 2020][8592468.732498] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:44 2020][8592468.737818] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592468.743000] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:44 2020][8592468.749038] [] ? at_measured+0x108/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.755573] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:44 2020][8592468.762812] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:44 2020][8592468.768563] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:44 2020][8592468.774414] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:44 2020][8592468.779729] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:44 2020][8592468.785910] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:44 2020][8592468.791345] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:44 2020][8592468.796752] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:44 2020][8592468.802759] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:44 2020][8592468.808093] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:44 2020][8592468.814970] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592468.821073] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:44 2020][8592468.826671] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:44 2020][8592468.833550] [] ? __getblk+0x2d/0x300 [Tue Mar 17 17:59:44 2020][8592468.838952] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:44 2020][8592468.846549] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.854816] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.862503] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:44 2020][8592468.869472] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.876614] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.884117] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.891089] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:44 2020][8592468.897882] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 17:59:44 2020][8592468.903994] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:44 2020][8592468.910720] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.918508] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.926528] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:44 2020][8592468.933010] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.940111] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.947860] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:44 2020][8592468.955128] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.962992] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.969955] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592468.975747] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.982222] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592468.989793] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592468.994882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.001148] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592469.007761] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.014041] ll_ost03_042 S ffff9c597f26e180 0 68627 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592469.021313] Call Trace: [Tue Mar 17 17:59:44 2020][8592469.023941] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:44 2020][8592469.030035] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592469.035226] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.042109] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592469.047890] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.054390] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.061963] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592469.067023] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.073305] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592469.079919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.086187] ll_ost02_040 D ffff9c597ee48000 0 68628 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592469.093481] Call Trace: [Tue Mar 17 17:59:44 2020][8592469.096111] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592469.101262] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:44 2020][8592469.108325] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:44 2020][8592469.114337] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:44 2020][8592469.121594] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:44 2020][8592469.128307] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.135795] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:44 2020][8592469.141999] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:44 2020][8592469.148790] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.156108] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.163698] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.170837] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:44 2020][8592469.177114] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:44 2020][8592469.183241] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:44 2020][8592469.189973] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.197745] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.205770] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:44 2020][8592469.212249] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.219332] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.227104] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:44 2020][8592469.234356] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.242229] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.249191] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:44 2020][8592469.254654] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.261126] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.268699] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592469.273767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.280036] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592469.286655] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.292935] ll_ost00_034 S ffff9c597ee49040 0 68629 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592469.300207] Call Trace: [Tue Mar 17 17:59:44 2020][8592469.302835] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:44 2020][8592469.308929] [] schedule+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592469.314126] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.321001] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:44 2020][8592469.326778] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.333266] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:44 2020][8592469.340831] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:44 2020][8592469.345884] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.352165] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:44 2020][8592469.358801] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:44 2020][8592469.365086] ll_ost02_041 D ffff9c597ee4a080 0 68630 2 0x00000080 [Tue Mar 17 17:59:44 2020][8592469.372358] Call Trace: [Tue Mar 17 17:59:44 2020][8592469.375003] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:44 2020][8592469.381616] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:44 2020][8592469.387997] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:44 2020][8592469.393319] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:44 2020][8592469.398819] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.405803] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:44 2020][8592469.413202] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:44 2020][8592469.419127] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:45 2020][8592469.425408] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:45 2020][8592469.430731] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.437696] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.445199] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.452516] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.460178] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.467330] [] evict+0xb4/0x180 [Tue Mar 17 17:59:45 2020][8592469.472299] [] iput+0xfc/0x190 [Tue Mar 17 17:59:45 2020][8592469.477190] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.484540] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:45 2020][8592469.491942] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:45 2020][8592469.499098] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:45 2020][8592469.505750] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:45 2020][8592469.512478] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.520250] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.528267] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:45 2020][8592469.534750] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.541850] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.549601] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:45 2020][8592469.556867] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.564733] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.571711] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:45 2020][8592469.577141] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.583645] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.591213] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592469.596266] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.602547] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592469.609163] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.615438] ll_ost01_041 D ffff9c597ee4b0c0 0 68631 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592469.622732] Call Trace: [Tue Mar 17 17:59:45 2020][8592469.625360] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:45 2020][8592469.631107] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:45 2020][8592469.637670] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592469.644290] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:45 2020][8592469.650651] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:45 2020][8592469.656018] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.663015] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.670108] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.677860] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:45 2020][8592469.685125] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.692981] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.699947] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:45 2020][8592469.705410] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.711885] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.719452] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592469.724520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.730792] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592469.737427] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.743695] ll_ost03_043 S ffff9c597ee4c100 0 68632 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592469.750965] Call Trace: [Tue Mar 17 17:59:45 2020][8592469.753607] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:45 2020][8592469.759703] [] schedule+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592469.764902] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.771782] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:45 2020][8592469.777573] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.784042] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.791631] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592469.796685] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.802974] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592469.809589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.815870] ll_ost00_035 D ffff9c597ee4d140 0 68633 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592469.823140] Call Trace: [Tue Mar 17 17:59:45 2020][8592469.825769] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592469.832383] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:45 2020][8592469.838757] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:45 2020][8592469.844107] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.851100] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.858194] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.865943] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:45 2020][8592469.873198] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.881071] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.888033] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:45 2020][8592469.893497] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.899968] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592469.907542] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592469.912615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.918886] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592469.925507] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592469.931796] ll_ost02_042 D ffff9c597ee4e180 0 68634 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592469.939068] Call Trace: [Tue Mar 17 17:59:45 2020][8592469.941695] [] schedule+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592469.946840] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:45 2020][8592469.953918] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:45 2020][8592469.959934] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:45 2020][8592469.967175] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:45 2020][8592469.973887] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:45 2020][8592469.981385] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:45 2020][8592469.987572] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:45 2020][8592469.994371] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.001701] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.009276] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.016418] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:45 2020][8592470.022718] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:45 2020][8592470.028817] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:45 2020][8592470.035548] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.043336] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.051343] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:45 2020][8592470.057818] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.064916] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.072664] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:45 2020][8592470.079930] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.087786] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.094751] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:45 2020][8592470.100562] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.107034] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.114606] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592470.119679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592470.125951] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592470.132563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592470.138852] ll_ost01_042 S ffff9c597f7d0000 0 68635 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592470.146124] Call Trace: [Tue Mar 17 17:59:45 2020][8592470.148751] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:45 2020][8592470.154847] [] schedule+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592470.160050] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.166927] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:45 2020][8592470.172711] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.179200] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.186773] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:45 2020][8592470.191839] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592470.198111] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:45 2020][8592470.204730] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592470.211012] ll_ost03_044 D ffff9c597f7d1040 0 68636 2 0x00000080 [Tue Mar 17 17:59:45 2020][8592470.218299] Call Trace: [Tue Mar 17 17:59:45 2020][8592470.220931] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:45 2020][8592470.226251] [] ? bh_lru_install+0x18a/0x1e0 [Tue Mar 17 17:59:45 2020][8592470.232294] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:45 2020][8592470.237612] [] schedule+0x29/0x70 [Tue Mar 17 17:59:45 2020][8592470.242751] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:45 2020][8592470.248772] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:45 2020][8592470.254087] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:45 2020][8592470.260268] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:45 2020][8592470.265687] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:45 2020][8592470.271105] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 17:59:45 2020][8592470.276683] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:45 2020][8592470.282018] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 17:59:45 2020][8592470.288460] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:45 2020][8592470.294558] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 17:59:45 2020][8592470.301456] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.309726] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:45 2020][8592470.317317] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.325586] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.332568] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.340319] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.347228] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.354363] [] evict+0xb4/0x180 [Tue Mar 17 17:59:45 2020][8592470.359335] [] iput+0xfc/0x190 [Tue Mar 17 17:59:45 2020][8592470.364234] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:45 2020][8592470.371569] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:45 2020][8592470.378973] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:45 2020][8592470.386139] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:45 2020][8592470.392759] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:45 2020][8592470.399492] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.407293] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:45 2020][8592470.415301] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:45 2020][8592470.421781] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.428873] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.436625] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:46 2020][8592470.443891] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.451750] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.458723] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:46 2020][8592470.464155] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.470644] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.478219] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.483309] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.489581] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.496202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.502491] ll_ost01_043 D ffff9c597f7d2080 0 68637 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592470.509761] Call Trace: [Tue Mar 17 17:59:46 2020][8592470.512392] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592470.517542] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:46 2020][8592470.524609] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:46 2020][8592470.530620] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592470.537877] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:46 2020][8592470.544595] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592470.552091] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:46 2020][8592470.558276] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592470.565086] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592470.572409] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:46 2020][8592470.580003] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592470.587147] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:46 2020][8592470.593435] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:46 2020][8592470.599538] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:46 2020][8592470.606269] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.614072] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.622082] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:46 2020][8592470.628560] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.635651] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.643403] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:46 2020][8592470.650670] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.658533] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.665497] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592470.671287] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.677776] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.685345] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.690410] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.696680] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.703308] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.709576] ll_ost00_036 S ffff9c597f7d30c0 0 68638 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592470.716845] Call Trace: [Tue Mar 17 17:59:46 2020][8592470.719489] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:46 2020][8592470.725584] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592470.730766] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.737654] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:46 2020][8592470.744269] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592470.750058] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.756527] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.764099] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.769166] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.775436] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.782057] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.788343] ll_ost02_043 S ffff9c597f7d4100 0 68639 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592470.795617] Call Trace: [Tue Mar 17 17:59:46 2020][8592470.798246] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:46 2020][8592470.804338] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592470.809536] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.816412] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592470.822187] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.828676] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.836250] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.841325] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.847593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.854220] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.860506] ll_ost03_045 S ffff9c597f7d5140 0 68640 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592470.867777] Call Trace: [Tue Mar 17 17:59:46 2020][8592470.870418] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:46 2020][8592470.876516] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592470.881707] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.888589] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592470.894372] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.900860] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.908426] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.913487] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.919782] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.926401] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.932676] ll_ost03_046 S ffff9c597f7d6180 0 68641 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592470.939960] Call Trace: [Tue Mar 17 17:59:46 2020][8592470.942590] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:46 2020][8592470.948682] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592470.953865] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.960740] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592470.966522] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.973009] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592470.980576] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592470.985658] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592470.991930] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592470.998543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592471.004822] ll_ost01_044 D ffff9c597f250000 0 68642 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592471.012093] Call Trace: [Tue Mar 17 17:59:46 2020][8592471.014723] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592471.019866] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.026932] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:46 2020][8592471.032944] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.040187] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.046905] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.054407] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:46 2020][8592471.060591] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.067402] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.074722] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.082310] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.089451] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:46 2020][8592471.095726] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:46 2020][8592471.101840] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:46 2020][8592471.108567] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.116361] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.124370] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:46 2020][8592471.130847] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.137940] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.145691] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:46 2020][8592471.152956] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.160813] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.167778] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:46 2020][8592471.173231] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.179719] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.187287] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592471.192362] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592471.198633] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592471.205253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592471.211533] ll_ost03_047 S ffff9c597f251040 0 68643 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592471.218804] Call Trace: [Tue Mar 17 17:59:46 2020][8592471.221435] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:46 2020][8592471.227538] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592471.232740] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.239618] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:46 2020][8592471.245443] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.251920] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.259491] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:46 2020][8592471.264565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592471.270833] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:46 2020][8592471.277447] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:46 2020][8592471.283725] ll_ost02_044 D ffff9c597f252080 0 68644 2 0x00000080 [Tue Mar 17 17:59:46 2020][8592471.290999] Call Trace: [Tue Mar 17 17:59:46 2020][8592471.293629] [] schedule+0x29/0x70 [Tue Mar 17 17:59:46 2020][8592471.298779] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.305859] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:46 2020][8592471.311874] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.319115] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.325822] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.333318] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:46 2020][8592471.339504] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:46 2020][8592471.346300] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.353633] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.361213] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:46 2020][8592471.368373] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:46 2020][8592471.374662] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:46 2020][8592471.380766] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:46 2020][8592471.387512] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.395281] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.403292] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:46 2020][8592471.409784] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.416864] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:46 2020][8592471.424613] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:47 2020][8592471.431880] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.439743] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.446765] [] ? ptlrpc_server_post_idle_rqbds+0x7e/0xf0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.454711] [] ? ptlrpc_grow_req_bufs+0x50/0x2a0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.461978] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.468447] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.476032] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.481088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.487369] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.493986] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.500258] ll_ost01_045 S ffff9c597f2530c0 0 68645 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.507553] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.510182] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592471.516275] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.521458] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.528330] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592471.534105] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.540595] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.548169] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.553232] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.559519] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.566150] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.572419] ll_ost03_048 S ffff9c597f254100 0 68646 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.579705] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.582333] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592471.588435] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.593628] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.600507] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592471.606282] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.612773] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.620346] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.625434] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.631708] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.638330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.644619] ll_ost03_049 D ffff9c597f255140 0 68647 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.651890] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.654518] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:47 2020][8592471.660526] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.667161] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:47 2020][8592471.673518] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:47 2020][8592471.678873] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.685902] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.692984] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.700749] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:47 2020][8592471.708004] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.715877] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.722838] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:47 2020][8592471.728267] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.734753] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.742323] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.747397] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.753682] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.760295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.766577] ll_ost00_037 S ffff9c597f256180 0 68648 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.773849] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.776476] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592471.782570] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.787760] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.794635] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592471.800418] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.806906] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.814479] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.819561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.825834] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.832469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.838738] ll_ost02_045 S ffff9c597e518000 0 68649 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.846006] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.848635] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592471.854751] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.859923] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.866802] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592471.872614] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.879085] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.886659] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592471.891733] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.898003] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592471.904637] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592471.910907] ll_ost02_046 D ffff9c597e519040 0 68650 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592471.918193] Call Trace: [Tue Mar 17 17:59:47 2020][8592471.920820] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592471.927434] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:47 2020][8592471.933824] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:47 2020][8592471.939177] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.946177] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.953270] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.961020] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:47 2020][8592471.968275] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.976151] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592471.983118] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:47 2020][8592471.989738] [] ? _raw_spin_lock+0x20/0x30 [Tue Mar 17 17:59:47 2020][8592471.995615] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.002096] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.009664] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592472.014739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.021009] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592472.027629] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.033909] ll_ost00_038 S ffff9c597e51a080 0 68651 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592472.041191] Call Trace: [Tue Mar 17 17:59:47 2020][8592472.043819] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592472.049912] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592472.055093] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.061967] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592472.067769] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.074240] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.081814] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592472.086886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.093159] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592472.099782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.106068] ll_ost03_050 S ffff9c597e51b0c0 0 68652 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592472.113341] Call Trace: [Tue Mar 17 17:59:47 2020][8592472.115970] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592472.122071] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592472.127268] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.134143] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592472.139920] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.146406] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.153972] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592472.159040] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.165309] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592472.171921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.178203] ll_ost01_046 D ffff9c597e51c100 0 68653 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592472.185490] Call Trace: [Tue Mar 17 17:59:47 2020][8592472.188121] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:47 2020][8592472.194135] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592472.200772] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:47 2020][8592472.207128] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:47 2020][8592472.212482] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.219492] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.226572] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.234334] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:47 2020][8592472.241602] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.249458] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.256453] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:47 2020][8592472.261890] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.268380] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.275949] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592472.281009] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.287291] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592472.293908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.300180] ll_ost00_039 S ffff9c597e51d140 0 68654 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592472.307467] Call Trace: [Tue Mar 17 17:59:47 2020][8592472.310096] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:47 2020][8592472.316188] [] schedule+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592472.321387] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.328261] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:47 2020][8592472.334891] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:47 2020][8592472.340666] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.347158] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.354729] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:47 2020][8592472.359796] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.366065] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:47 2020][8592472.372679] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:47 2020][8592472.378980] ll_ost02_047 D ffff9c597e51e180 0 68655 2 0x00000080 [Tue Mar 17 17:59:47 2020][8592472.386255] Call Trace: [Tue Mar 17 17:59:47 2020][8592472.388888] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:47 2020][8592472.394639] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:47 2020][8592472.401180] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:47 2020][8592472.407797] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:47 2020][8592472.414157] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:47 2020][8592472.419527] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:47 2020][8592472.426524] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.433610] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.441387] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:48 2020][8592472.448636] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.456512] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.463480] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:48 2020][8592472.469264] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.475752] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.483318] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592472.488370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.494651] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592472.501282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.507550] ll_ost02_048 D ffff9c597e5a0000 0 68656 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592472.514846] Call Trace: [Tue Mar 17 17:59:48 2020][8592472.517474] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592472.524094] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:48 2020][8592472.530463] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:48 2020][8592472.535780] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:48 2020][8592472.541279] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.548262] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.555658] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:48 2020][8592472.561579] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:48 2020][8592472.567880] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:48 2020][8592472.573208] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.580174] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.587675] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.594993] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.602654] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.609808] [] evict+0xb4/0x180 [Tue Mar 17 17:59:48 2020][8592472.614774] [] iput+0xfc/0x190 [Tue Mar 17 17:59:48 2020][8592472.619662] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.627021] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:48 2020][8592472.634430] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:48 2020][8592472.641582] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:48 2020][8592472.648222] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:48 2020][8592472.654955] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.662741] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.670751] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:48 2020][8592472.677239] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.684321] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.692082] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:48 2020][8592472.699332] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.707219] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.714183] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:48 2020][8592472.719632] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.726102] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.733672] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592472.738741] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.745009] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592472.751621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.757904] ll_ost00_040 S ffff9c597e5a1040 0 68657 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592472.765190] Call Trace: [Tue Mar 17 17:59:48 2020][8592472.767819] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:48 2020][8592472.773912] [] schedule+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592472.779097] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.785977] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:48 2020][8592472.791751] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.798240] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.805807] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592472.810881] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.817152] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592472.823774] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.830075] ll_ost00_041 S ffff9c597e5a2080 0 68658 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592472.837352] Call Trace: [Tue Mar 17 17:59:48 2020][8592472.839993] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:48 2020][8592472.846091] [] schedule+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592472.851283] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.858163] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:48 2020][8592472.863945] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.870434] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592472.878000] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592472.883082] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.889355] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592472.895974] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592472.902255] ll_ost02_049 D ffff9c597e5a30c0 0 68659 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592472.909526] Call Trace: [Tue Mar 17 17:59:48 2020][8592472.912157] [] schedule+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592472.917312] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:48 2020][8592472.924372] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:48 2020][8592472.930386] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:48 2020][8592472.937625] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:48 2020][8592472.944335] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.951843] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:48 2020][8592472.958032] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:48 2020][8592472.964829] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.972160] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.979737] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:48 2020][8592472.986877] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:48 2020][8592472.993165] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:48 2020][8592472.999268] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:48 2020][8592473.006000] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.013800] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.021810] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:48 2020][8592473.028302] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.035386] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.043147] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:48 2020][8592473.050401] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.058277] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.065236] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:48 2020][8592473.070678] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.077149] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.084718] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592473.089801] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592473.096073] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592473.102695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592473.108983] ll_ost01_047 D ffff9c597e5a4100 0 68660 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592473.116254] Call Trace: [Tue Mar 17 17:59:48 2020][8592473.118885] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:48 2020][8592473.125073] [] ? jbd2_journal_add_journal_head+0x65/0x150 [jbd2] [Tue Mar 17 17:59:48 2020][8592473.132919] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592473.139536] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:48 2020][8592473.145925] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:48 2020][8592473.151244] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:48 2020][8592473.156737] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.163719] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.171117] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:48 2020][8592473.177051] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:48 2020][8592473.183322] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:48 2020][8592473.188650] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.195630] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.203122] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.210462] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.218123] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.225275] [] evict+0xb4/0x180 [Tue Mar 17 17:59:48 2020][8592473.230242] [] iput+0xfc/0x190 [Tue Mar 17 17:59:48 2020][8592473.235155] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:48 2020][8592473.242481] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:48 2020][8592473.249907] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:48 2020][8592473.257074] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:48 2020][8592473.263692] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:48 2020][8592473.270452] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.278236] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.286241] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:48 2020][8592473.292733] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.299816] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.307577] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:48 2020][8592473.314835] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.322709] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.329676] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:48 2020][8592473.335127] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.341603] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.349177] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592473.354266] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592473.360539] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:48 2020][8592473.367174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:48 2020][8592473.373443] ll_ost03_051 S ffff9c597e5a5140 0 68661 2 0x00000080 [Tue Mar 17 17:59:48 2020][8592473.380721] Call Trace: [Tue Mar 17 17:59:48 2020][8592473.383362] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:48 2020][8592473.389457] [] schedule+0x29/0x70 [Tue Mar 17 17:59:48 2020][8592473.394642] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.401523] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:48 2020][8592473.407298] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.413786] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:48 2020][8592473.421352] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:48 2020][8592473.426418] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.432687] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.439300] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.445582] ll_ost00_042 S ffff9c597e5a6180 0 68662 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592473.452851] Call Trace: [Tue Mar 17 17:59:49 2020][8592473.455481] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592473.461575] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592473.466782] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.473657] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592473.479440] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.485930] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.493502] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592473.498570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.504839] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.511459] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.517741] ll_ost03_052 D ffff9c597e938000 0 68663 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592473.525011] Call Trace: [Tue Mar 17 17:59:49 2020][8592473.527640] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:49 2020][8592473.533388] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:49 2020][8592473.539934] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592473.546552] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:49 2020][8592473.552914] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:49 2020][8592473.558282] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.565279] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.572372] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.580124] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:49 2020][8592473.587391] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.595253] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.602233] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:49 2020][8592473.607662] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.614151] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.621719] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592473.626794] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.633063] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.639677] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.645965] ll_ost00_043 S ffff9c597e939040 0 68664 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592473.653237] Call Trace: [Tue Mar 17 17:59:49 2020][8592473.655865] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592473.661960] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592473.667165] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.674040] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592473.679822] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.686310] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.693878] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592473.698939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.705230] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.711846] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.718118] ll_ost01_048 D ffff9c597e93a080 0 68665 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592473.725403] Call Trace: [Tue Mar 17 17:59:49 2020][8592473.728032] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592473.733175] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:49 2020][8592473.740244] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:49 2020][8592473.746255] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:49 2020][8592473.753510] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:49 2020][8592473.760221] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592473.767724] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:49 2020][8592473.773909] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:49 2020][8592473.780705] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592473.788036] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:49 2020][8592473.795615] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592473.802753] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:49 2020][8592473.809042] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:49 2020][8592473.815147] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:49 2020][8592473.821878] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.829664] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.837671] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:49 2020][8592473.844163] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.851244] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.859020] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:49 2020][8592473.866271] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.874130] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.881110] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:49 2020][8592473.886541] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.893030] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.900596] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592473.905664] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.911933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.918552] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.924846] ll_ost01_049 S ffff9c597e93b0c0 0 68666 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592473.932113] Call Trace: [Tue Mar 17 17:59:49 2020][8592473.934743] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592473.940837] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592473.946028] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.952908] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592473.958683] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.965170] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592473.972739] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592473.977829] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.984099] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592473.990726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592473.996995] ll_ost02_050 S ffff9c597e93c100 0 68667 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592474.004265] Call Trace: [Tue Mar 17 17:59:49 2020][8592474.006910] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592474.013002] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592474.018187] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.025068] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592474.030852] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.037353] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.044925] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592474.049986] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.056276] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592474.062893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.069164] ll_ost00_044 S ffff9c597e93d140 0 68668 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592474.076451] Call Trace: [Tue Mar 17 17:59:49 2020][8592474.079078] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592474.085173] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592474.090363] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.097251] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592474.103029] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.109516] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.117083] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592474.122157] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.128427] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592474.135049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.141338] ll_ost03_053 S ffff9c597e93e180 0 68669 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592474.148609] Call Trace: [Tue Mar 17 17:59:49 2020][8592474.151240] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:49 2020][8592474.157340] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592474.162545] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.169423] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592474.175205] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.181691] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.189261] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592474.194334] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.200604] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592474.207217] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.213505] ll_ost02_051 D ffff9c5991f70000 0 68670 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592474.220794] Call Trace: [Tue Mar 17 17:59:49 2020][8592474.223422] [] schedule+0x29/0x70 [Tue Mar 17 17:59:49 2020][8592474.228567] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:49 2020][8592474.235650] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:49 2020][8592474.241659] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:49 2020][8592474.248910] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:49 2020][8592474.255618] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592474.263113] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:49 2020][8592474.269298] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:49 2020][8592474.276112] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592474.283456] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:49 2020][8592474.291028] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:49 2020][8592474.298199] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:49 2020][8592474.304468] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:49 2020][8592474.310571] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:49 2020][8592474.317316] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.325085] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.333096] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:49 2020][8592474.339587] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.346670] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.354446] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:49 2020][8592474.361695] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.369572] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.376538] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:49 2020][8592474.382314] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.388802] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:49 2020][8592474.396377] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:49 2020][8592474.401438] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.407719] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:49 2020][8592474.414333] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:49 2020][8592474.420601] ll_ost01_050 D ffff9c5991f71040 0 68671 2 0x00000080 [Tue Mar 17 17:59:49 2020][8592474.427904] Call Trace: [Tue Mar 17 17:59:49 2020][8592474.430533] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592474.435685] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:50 2020][8592474.442750] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:50 2020][8592474.448761] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:50 2020][8592474.456003] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:50 2020][8592474.462712] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592474.470214] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:50 2020][8592474.476399] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:50 2020][8592474.483196] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592474.490543] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:50 2020][8592474.498123] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592474.505274] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:50 2020][8592474.511551] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:50 2020][8592474.517668] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:50 2020][8592474.524402] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.532177] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.540201] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:50 2020][8592474.546681] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.553789] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.561535] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:50 2020][8592474.568802] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.576664] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.583644] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:50 2020][8592474.589075] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.595563] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.603130] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592474.608198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.614465] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592474.621080] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.627380] ll_ost00_045 S ffff9c5991f72080 0 68672 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592474.634647] Call Trace: [Tue Mar 17 17:59:50 2020][8592474.637291] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592474.643387] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592474.648570] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.655452] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592474.661235] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.667722] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.675289] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592474.680341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.686636] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592474.693254] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.699522] ll_ost03_054 S ffff9c5991f730c0 0 68673 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592474.706816] Call Trace: [Tue Mar 17 17:59:50 2020][8592474.709446] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592474.715545] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592474.720729] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.727602] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592474.733385] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.739889] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.747463] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592474.752532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.758803] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592474.765439] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.771706] ll_ost01_051 S ffff9c5991f74100 0 68674 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592474.778992] Call Trace: [Tue Mar 17 17:59:50 2020][8592474.781640] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 17:59:50 2020][8592474.788953] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592474.794108] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 17:59:50 2020][8592474.800131] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 17:59:50 2020][8592474.806687] [] ldlm_completion_ast+0x430/0x860 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.813780] [] ? ldlm_completion_ast_async+0x300/0x300 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.821546] [] ? ldlm_lock_enqueue+0x3ff/0xa20 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.828611] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592474.834384] [] ldlm_cli_enqueue_local+0x231/0x830 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.841726] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.849774] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.857519] [] ofd_destroy_by_fid+0x1dd/0x4a0 [ofd] [Tue Mar 17 17:59:50 2020][8592474.864263] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.872046] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.880052] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:50 2020][8592474.886546] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.893626] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.901373] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:50 2020][8592474.908641] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.916503] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.923485] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:50 2020][8592474.928916] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.935403] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.942984] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592474.948038] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.954320] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592474.960934] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592474.967202] ll_ost00_046 S ffff9c5991f75140 0 68675 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592474.974495] Call Trace: [Tue Mar 17 17:59:50 2020][8592474.977125] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592474.983218] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592474.988410] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592474.995290] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592475.001065] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.007566] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.015138] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592475.020211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.026483] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592475.033117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.039386] ll_ost01_052 D ffff9c5991f76180 0 68676 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592475.046654] Call Trace: [Tue Mar 17 17:59:50 2020][8592475.049297] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592475.054444] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:50 2020][8592475.061525] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:50 2020][8592475.067539] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:50 2020][8592475.074783] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:50 2020][8592475.081499] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592475.088993] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:50 2020][8592475.095176] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:50 2020][8592475.101989] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592475.109311] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:50 2020][8592475.116913] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:50 2020][8592475.124058] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:50 2020][8592475.130331] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:50 2020][8592475.136459] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:50 2020][8592475.143188] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.150962] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.158980] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:50 2020][8592475.165459] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.172553] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.180303] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:50 2020][8592475.187573] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.195434] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.202427] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:50 2020][8592475.207864] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.214348] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.221917] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592475.226991] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.233262] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592475.239905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.246190] ll_ost03_055 S ffff9c597dc78000 0 68677 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592475.253462] Call Trace: [Tue Mar 17 17:59:50 2020][8592475.256103] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592475.262200] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592475.267405] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.274281] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592475.280065] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.286554] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.294130] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592475.299189] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.305469] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592475.312100] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.318369] ll_ost03_056 S ffff9c597dc79040 0 68678 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592475.325678] Call Trace: [Tue Mar 17 17:59:50 2020][8592475.328310] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592475.334412] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592475.339601] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.346476] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592475.352259] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.358746] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.366314] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:50 2020][8592475.371391] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.377658] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:50 2020][8592475.384270] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:50 2020][8592475.390564] ll_ost01_053 S ffff9c597dc7a080 0 68679 2 0x00000080 [Tue Mar 17 17:59:50 2020][8592475.397840] Call Trace: [Tue Mar 17 17:59:50 2020][8592475.400468] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:50 2020][8592475.406562] [] schedule+0x29/0x70 [Tue Mar 17 17:59:50 2020][8592475.411752] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.418627] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:50 2020][8592475.424425] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:50 2020][8592475.430902] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.438472] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.443553] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.449827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.456461] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.462729] ll_ost03_057 S ffff9c597dc7b0c0 0 68680 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.469999] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.472640] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.478737] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.483924] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.490801] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.496576] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.503078] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.510647] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.515700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.521983] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.528597] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.534879] ll_ost01_054 S ffff9c597dc7c100 0 68681 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.542150] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.544780] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.550880] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.556063] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.562959] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.568735] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.575225] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.582799] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.587851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.594132] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.600748] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.607014] ll_ost00_047 S ffff9c597dc7d140 0 68682 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.614301] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.616929] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.623023] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.628226] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.635103] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.640887] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.647375] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.654940] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.659992] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.666275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.672889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.679156] ll_ost02_052 S ffff9c597dc7e180 0 68683 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.686458] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.689092] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.695190] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.700384] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.707267] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.713054] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.719533] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.727104] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.732164] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.738441] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.745059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.751333] ll_ost02_053 S ffff9c597d568000 0 68684 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.758611] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.761241] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.767341] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.772530] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.779406] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.785197] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.791674] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.799247] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.804305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.810580] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.817215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.823483] ll_ost00_048 S ffff9c597d569040 0 68685 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.830769] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.833401] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.839499] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.844695] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.851574] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.857356] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.863857] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.871427] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.876494] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.882763] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.889377] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.895665] ll_ost01_055 S ffff9c597d56a080 0 68686 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.902937] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.905583] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592475.911677] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.916854] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.923746] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592475.929524] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.936003] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592475.943569] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592475.948622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.954902] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592475.961517] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592475.967783] ll_ost01_056 D ffff9c597d56b0c0 0 68687 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592475.975070] Call Trace: [Tue Mar 17 17:59:51 2020][8592475.977699] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592475.982849] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:51 2020][8592475.989930] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:51 2020][8592475.995944] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.003174] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.009888] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.017391] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:51 2020][8592476.023575] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.030372] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.037692] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.045269] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.052410] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:51 2020][8592476.058701] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:51 2020][8592476.064803] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:51 2020][8592476.071537] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.079325] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.087330] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:51 2020][8592476.093811] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.100896] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.108649] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:51 2020][8592476.115917] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.123778] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.130761] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:51 2020][8592476.136190] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.142665] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.150235] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592476.155288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592476.161556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592476.168177] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592476.174450] ll_ost00_049 S ffff9c597d56c100 0 68688 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592476.181721] Call Trace: [Tue Mar 17 17:59:51 2020][8592476.184349] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:51 2020][8592476.190450] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592476.195627] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.202507] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:51 2020][8592476.209126] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:51 2020][8592476.214903] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.221387] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.228958] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:51 2020][8592476.234019] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592476.240322] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:51 2020][8592476.246940] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:51 2020][8592476.253229] ll_ost00_050 D ffff9c597d56d140 0 68689 2 0x00000080 [Tue Mar 17 17:59:51 2020][8592476.260499] Call Trace: [Tue Mar 17 17:59:51 2020][8592476.263128] [] schedule+0x29/0x70 [Tue Mar 17 17:59:51 2020][8592476.268275] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.275322] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:51 2020][8592476.281331] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.288564] [] ? kiblnd_send+0x357/0xa20 [ko2iblnd] [Tue Mar 17 17:59:51 2020][8592476.295268] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.302004] [] ? lu_obj_hop_keycmp+0x1d/0x30 [obdclass] [Tue Mar 17 17:59:51 2020][8592476.309075] [] ? cfs_hash_bd_lookup_intent+0x63/0x170 [libcfs] [Tue Mar 17 17:59:51 2020][8592476.316733] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:51 2020][8592476.322926] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:51 2020][8592476.329726] [] ? ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.337047] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.344619] [] ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.351757] [] dqget+0x3fa/0x450 [Tue Mar 17 17:59:51 2020][8592476.356830] [] dquot_get_dqblk+0x14/0x1f0 [Tue Mar 17 17:59:51 2020][8592476.362679] [] osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 17:59:51 2020][8592476.370342] [] lquotactl_slv+0x27d/0x9d0 [lquota] [Tue Mar 17 17:59:51 2020][8592476.376907] [] ofd_quotactl+0x13c/0x380 [ofd] [Tue Mar 17 17:59:51 2020][8592476.383129] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.390215] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.397962] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:51 2020][8592476.405215] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:51 2020][8592476.413088] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.420049] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:52 2020][8592476.425476] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.431949] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.439523] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.444600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.450868] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.457491] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.463778] ll_ost01_057 S ffff9c597d56e180 0 68690 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592476.471049] Call Trace: [Tue Mar 17 17:59:52 2020][8592476.473678] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592476.479784] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592476.484958] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.491836] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592476.497632] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.504102] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.511686] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.516742] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.523008] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.529622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.535889] ll_ost02_054 D ffff9c597dd38000 0 68691 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592476.543166] Call Trace: [Tue Mar 17 17:59:52 2020][8592476.545793] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592476.550939] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:52 2020][8592476.558002] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:52 2020][8592476.564014] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:52 2020][8592476.571268] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:52 2020][8592476.577985] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:52 2020][8592476.585474] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:52 2020][8592476.591663] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:52 2020][8592476.598473] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:52 2020][8592476.605793] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:52 2020][8592476.613366] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:52 2020][8592476.620506] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:52 2020][8592476.626795] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:52 2020][8592476.632919] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:52 2020][8592476.639648] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.647436] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.655441] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:52 2020][8592476.661939] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.669023] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.676785] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:52 2020][8592476.684040] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.691904] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.698875] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:52 2020][8592476.704339] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.710811] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.718398] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.723454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.729735] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.736351] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.742637] ll_ost03_058 S ffff9c597dd39040 0 68692 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592476.749911] Call Trace: [Tue Mar 17 17:59:52 2020][8592476.752552] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592476.758648] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592476.763845] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.770721] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592476.776510] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.782979] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.790565] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.795621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.801904] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.808516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.814800] ll_ost01_058 S ffff9c597dd3a080 0 68693 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592476.822070] Call Trace: [Tue Mar 17 17:59:52 2020][8592476.824699] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592476.830800] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592476.835992] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.842871] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592476.848647] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.855135] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.862703] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.867777] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.874047] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.880666] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.886961] ll_ost00_051 D ffff9c597dd3b0c0 0 68694 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592476.894238] Call Trace: [Tue Mar 17 17:59:52 2020][8592476.896864] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592476.903494] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:52 2020][8592476.909850] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:52 2020][8592476.915204] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.922212] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.929294] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.937040] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:52 2020][8592476.944307] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.952162] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.959159] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:52 2020][8592476.964592] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.971079] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592476.978656] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592476.983714] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592476.989996] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592476.996612] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.002884] ll_ost02_055 S ffff9c597dd3c100 0 68695 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592477.010170] Call Trace: [Tue Mar 17 17:59:52 2020][8592477.012799] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592477.018893] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592477.024097] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.030976] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592477.036759] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.043246] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.050814] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592477.055889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.062159] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592477.068779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.075070] ll_ost00_052 S ffff9c597dd3d140 0 68696 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592477.082340] Call Trace: [Tue Mar 17 17:59:52 2020][8592477.084968] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592477.091060] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592477.096243] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.103119] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 17:59:52 2020][8592477.109738] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592477.115526] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.121996] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.129584] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592477.134638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.140938] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592477.147559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.153824] ll_ost01_059 S ffff9c597dd3e180 0 68697 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592477.161120] Call Trace: [Tue Mar 17 17:59:52 2020][8592477.163748] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592477.169843] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592477.175033] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.181915] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:52 2020][8592477.187689] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.194192] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.201762] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592477.206824] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.213111] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592477.219727] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.225992] ll_ost_io00_019 D ffff9c597f928000 0 68698 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592477.233281] Call Trace: [Tue Mar 17 17:59:52 2020][8592477.235908] [] schedule+0x29/0x70 [Tue Mar 17 17:59:52 2020][8592477.241065] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:52 2020][8592477.248155] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:52 2020][8592477.255296] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:52 2020][8592477.261341] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:52 2020][8592477.267527] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:52 2020][8592477.274324] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:52 2020][8592477.280555] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.287287] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.293936] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.301006] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:52 2020][8592477.307044] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.314486] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.321581] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.329331] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:52 2020][8592477.336597] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.344452] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.351433] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:52 2020][8592477.356865] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.363352] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:52 2020][8592477.370920] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:52 2020][8592477.375995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.382262] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:52 2020][8592477.388910] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:52 2020][8592477.395176] ll_ost02_056 S ffff9c597f929040 0 68699 2 0x00000080 [Tue Mar 17 17:59:52 2020][8592477.402446] Call Trace: [Tue Mar 17 17:59:52 2020][8592477.405088] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:52 2020][8592477.411185] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592477.416376] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.423258] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:53 2020][8592477.429040] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.435529] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.443096] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592477.448185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.454458] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592477.461079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.467367] ll_ost03_059 D ffff9c597f92a080 0 68700 2 0x00000080 [Tue Mar 17 17:59:53 2020][8592477.474640] Call Trace: [Tue Mar 17 17:59:53 2020][8592477.477267] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592477.482413] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.489476] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:53 2020][8592477.495488] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.502730] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.509437] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.516946] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:53 2020][8592477.523136] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.529945] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.537266] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.544855] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.551998] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:53 2020][8592477.558273] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:53 2020][8592477.564387] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:53 2020][8592477.571120] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.578894] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.586936] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:53 2020][8592477.593409] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.600501] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.608254] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:53 2020][8592477.615520] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.623384] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.630349] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:53 2020][8592477.635801] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.642278] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.649876] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592477.654936] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.661225] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592477.667841] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.674115] ll_ost_io01_018 D ffff9c597f92b0c0 0 68701 2 0x00000080 [Tue Mar 17 17:59:53 2020][8592477.681401] Call Trace: [Tue Mar 17 17:59:53 2020][8592477.684031] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592477.689181] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.696246] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:53 2020][8592477.702259] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.709514] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.716225] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.723981] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:53 2020][8592477.730165] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.736976] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.744297] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.751885] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.759028] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:53 2020][8592477.765302] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:53 2020][8592477.772109] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:53 2020][8592477.778324] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.785081] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.791728] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.798793] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:53 2020][8592477.804828] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.812276] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.819358] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.827120] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:53 2020][8592477.834376] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.842249] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.849223] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:53 2020][8592477.854671] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.861147] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.868718] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592477.873785] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.880055] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592477.886676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592477.892964] ll_ost_io03_019 D ffff9c597f92c100 0 68702 2 0x00000080 [Tue Mar 17 17:59:53 2020][8592477.900237] Call Trace: [Tue Mar 17 17:59:53 2020][8592477.902862] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592477.908008] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.915088] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:53 2020][8592477.921101] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.928344] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.935055] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.942382] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:53 2020][8592477.948568] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592477.955366] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.962695] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.970271] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592477.977412] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:53 2020][8592477.984235] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:53 2020][8592477.990449] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592477.997153] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 17:59:53 2020][8592478.003124] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.009772] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.016826] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:53 2020][8592478.022596] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:53 2020][8592478.029146] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.036574] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.043679] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.051425] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:53 2020][8592478.058693] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.066555] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.073523] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:53 2020][8592478.078972] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.085440] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.093013] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592478.098089] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592478.104357] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592478.110998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592478.117270] ll_ost_io02_021 D ffff9c597f92d140 0 68703 2 0x00000080 [Tue Mar 17 17:59:53 2020][8592478.124539] Call Trace: [Tue Mar 17 17:59:53 2020][8592478.127183] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592478.132329] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:53 2020][8592478.139395] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:53 2020][8592478.145406] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592478.152650] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:53 2020][8592478.158493] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:53 2020][8592478.165221] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592478.172531] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:53 2020][8592478.178749] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:53 2020][8592478.185544] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592478.192861] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:53 2020][8592478.200449] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:53 2020][8592478.207591] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:53 2020][8592478.214399] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:53 2020][8592478.220620] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.227351] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.234013] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.241094] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:53 2020][8592478.248451] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:53 2020][8592478.254207] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:53 2020][8592478.260782] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.268207] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.275288] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.283064] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:53 2020][8592478.290315] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.298189] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.305148] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:53 2020][8592478.310577] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.317078] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.324649] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592478.329724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592478.335994] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592478.342615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592478.348896] ll_ost01_060 S ffff9c597f92e180 0 68704 2 0x00000080 [Tue Mar 17 17:59:53 2020][8592478.356166] Call Trace: [Tue Mar 17 17:59:53 2020][8592478.358795] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:53 2020][8592478.364889] [] schedule+0x29/0x70 [Tue Mar 17 17:59:53 2020][8592478.370093] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.376969] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:53 2020][8592478.382742] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.389232] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:53 2020][8592478.396800] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:53 2020][8592478.401862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:53 2020][8592478.408148] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:53 2020][8592478.414767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592478.421055] ll_ost_io00_020 D ffff9c597dec0000 0 68705 2 0x00000080 [Tue Mar 17 17:59:54 2020][8592478.428325] Call Trace: [Tue Mar 17 17:59:54 2020][8592478.430969] [] schedule+0x29/0x70 [Tue Mar 17 17:59:54 2020][8592478.436115] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.443194] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:54 2020][8592478.449210] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.456450] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:54 2020][8592478.462287] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.469018] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.476324] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:54 2020][8592478.482521] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.489313] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.496629] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.504218] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.511361] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:54 2020][8592478.518171] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:54 2020][8592478.524388] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.531119] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.537780] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.544827] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:54 2020][8592478.550368] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.557451] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.565226] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:54 2020][8592478.572477] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.580351] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.587311] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:54 2020][8592478.592738] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.599226] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.606795] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:54 2020][8592478.611869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592478.618139] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:54 2020][8592478.624759] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592478.631052] ll_ost_io01_019 D ffff9c597dec1040 0 68706 2 0x00000080 [Tue Mar 17 17:59:54 2020][8592478.638329] Call Trace: [Tue Mar 17 17:59:54 2020][8592478.640958] [] schedule+0x29/0x70 [Tue Mar 17 17:59:54 2020][8592478.646118] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.653169] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:54 2020][8592478.659179] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.666419] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:54 2020][8592478.672258] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.678985] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.686296] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:54 2020][8592478.692513] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592478.699309] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.706624] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.714212] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.721353] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:54 2020][8592478.728150] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:54 2020][8592478.734379] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.741113] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.747776] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.754850] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:54 2020][8592478.762192] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:54 2020][8592478.767944] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:54 2020][8592478.773471] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.780564] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.788314] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:54 2020][8592478.795583] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.803448] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.810425] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:54 2020][8592478.815856] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.822342] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.829912] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:54 2020][8592478.834999] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592478.841271] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:54 2020][8592478.847885] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592478.854165] ll_ost_io02_022 D ffff9c597dec2080 0 68707 2 0x00000080 [Tue Mar 17 17:59:54 2020][8592478.861436] Call Trace: [Tue Mar 17 17:59:54 2020][8592478.864067] [] schedule+0x29/0x70 [Tue Mar 17 17:59:54 2020][8592478.869222] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.876279] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592478.883418] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:54 2020][8592478.889445] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:54 2020][8592478.895633] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:54 2020][8592478.902428] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:54 2020][8592478.908677] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.915414] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.922062] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.929152] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:54 2020][8592478.936464] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:54 2020][8592478.942216] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:54 2020][8592478.947766] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.954863] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.962613] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:54 2020][8592478.969865] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.977726] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.984707] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:54 2020][8592478.990147] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592478.996619] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.004194] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:54 2020][8592479.009250] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592479.015519] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:54 2020][8592479.022153] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592479.028421] ll_ost_io03_020 D ffff9c597dec30c0 0 68708 2 0x00000080 [Tue Mar 17 17:59:54 2020][8592479.035691] Call Trace: [Tue Mar 17 17:59:54 2020][8592479.038334] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:54 2020][8592479.044951] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:54 2020][8592479.051324] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:54 2020][8592479.056642] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:54 2020][8592479.062158] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.069127] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.076524] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 17:59:54 2020][8592479.082834] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.091189] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 17:59:54 2020][8592479.097569] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.104800] [] ? ldiskfs_es_insert_extent+0xdb/0x1c0 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.112457] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 17:59:54 2020][8592479.118057] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.125720] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.133383] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.141319] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.148724] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.155793] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.164010] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592479.170979] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.178222] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.185536] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 17:59:54 2020][8592479.192330] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:54 2020][8592479.198567] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.205306] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.211969] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.219023] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:54 2020][8592479.225088] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.232520] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.239601] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.247377] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:54 2020][8592479.254626] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.262516] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.269478] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:54 2020][8592479.274936] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.281407] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:54 2020][8592479.288978] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:54 2020][8592479.294066] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592479.300340] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:54 2020][8592479.306952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:54 2020][8592479.313232] ll_ost_io01_020 D ffff9c597dec4100 0 68709 2 0x00000080 [Tue Mar 17 17:59:54 2020][8592479.320505] Call Trace: [Tue Mar 17 17:59:54 2020][8592479.323132] [] schedule+0x29/0x70 [Tue Mar 17 17:59:54 2020][8592479.328276] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:54 2020][8592479.335343] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:54 2020][8592479.341352] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592479.348598] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 17:59:54 2020][8592479.354439] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:54 2020][8592479.361182] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.368497] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:54 2020][8592479.374701] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:54 2020][8592479.381500] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.388833] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.396407] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:54 2020][8592479.403547] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 17:59:54 2020][8592479.410355] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:54 2020][8592479.416568] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.423270] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:55 2020][8592479.428821] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.435900] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.443668] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:55 2020][8592479.450919] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.458781] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.465759] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:55 2020][8592479.471190] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.477677] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.485245] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592479.490333] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.496608] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592479.503227] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.509509] ll_ost02_057 D ffff9c597dec5140 0 68710 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592479.516780] Call Trace: [Tue Mar 17 17:59:55 2020][8592479.519408] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592479.526022] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:55 2020][8592479.532397] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:55 2020][8592479.537712] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:55 2020][8592479.543206] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.550202] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.557604] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 17:59:55 2020][8592479.563544] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 17:59:55 2020][8592479.569818] [] ? __brelse+0x3d/0x50 [Tue Mar 17 17:59:55 2020][8592479.575142] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.582124] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.589616] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.596944] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.604609] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.611761] [] evict+0xb4/0x180 [Tue Mar 17 17:59:55 2020][8592479.616742] [] iput+0xfc/0x190 [Tue Mar 17 17:59:55 2020][8592479.621630] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.628982] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 17:59:55 2020][8592479.636381] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 17:59:55 2020][8592479.643535] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 17:59:55 2020][8592479.650174] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 17:59:55 2020][8592479.656910] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.664679] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.672699] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:55 2020][8592479.679178] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.686273] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.694022] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:55 2020][8592479.701288] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.709143] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.716122] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:55 2020][8592479.721556] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.728042] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.735608] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592479.740676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.746947] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592479.753567] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.759872] ll_ost00_053 S ffff9c597dec6180 0 68711 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592479.767143] Call Trace: [Tue Mar 17 17:59:55 2020][8592479.769776] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:55 2020][8592479.775874] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592479.781057] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.787932] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:55 2020][8592479.793714] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.800203] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.807777] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592479.812831] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.819133] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592479.825752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592479.832041] ll_ost_io00_021 D ffff9c597d4f8000 0 68712 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592479.839313] Call Trace: [Tue Mar 17 17:59:55 2020][8592479.841950] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.849614] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592479.854777] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:55 2020][8592479.861831] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:55 2020][8592479.867854] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:55 2020][8592479.875091] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:55 2020][8592479.881798] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.889555] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:55 2020][8592479.895740] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:55 2020][8592479.902536] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.909866] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.917443] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:55 2020][8592479.924583] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:55 2020][8592479.930871] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:55 2020][8592479.937671] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:55 2020][8592479.943917] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.950623] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:55 2020][8592479.956391] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 17:59:55 2020][8592479.962947] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.970393] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.977478] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592479.985241] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:55 2020][8592479.992494] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.000371] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.007338] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:55 2020][8592480.013121] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.019623] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.027193] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592480.032269] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.038538] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592480.045161] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.051447] ll_ost01_061 S ffff9c597d4f9040 0 68713 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592480.058718] Call Trace: [Tue Mar 17 17:59:55 2020][8592480.061361] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:55 2020][8592480.067457] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592480.072655] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.079532] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:55 2020][8592480.085313] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.091801] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.099368] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592480.104435] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.110707] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592480.117326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.123607] ll_ost_io03_021 D ffff9c597d4fa080 0 68714 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592480.130877] Call Trace: [Tue Mar 17 17:59:55 2020][8592480.133508] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592480.138663] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 17:59:55 2020][8592480.145745] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592480.152886] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:55 2020][8592480.158908] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 17:59:55 2020][8592480.165114] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 17:59:55 2020][8592480.171907] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:55 2020][8592480.178120] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.184868] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.191517] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.198615] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:55 2020][8592480.205926] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:55 2020][8592480.211673] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:55 2020][8592480.217212] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.224297] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.232056] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:55 2020][8592480.239306] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.247194] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.254156] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:55 2020][8592480.259600] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.266089] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.273657] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592480.278732] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.285002] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592480.291623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.297911] ll_ost03_060 S ffff9c597d4fb0c0 0 68715 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592480.305184] Call Trace: [Tue Mar 17 17:59:55 2020][8592480.307812] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:55 2020][8592480.313908] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592480.319098] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.325992] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:55 2020][8592480.331770] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.338258] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:55 2020][8592480.345825] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:55 2020][8592480.350901] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.357170] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:55 2020][8592480.363791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:55 2020][8592480.370079] ll_ost_io00_022 D ffff9c597d4fc100 0 68716 2 0x00000080 [Tue Mar 17 17:59:55 2020][8592480.377350] Call Trace: [Tue Mar 17 17:59:55 2020][8592480.380002] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:55 2020][8592480.387659] [] schedule+0x29/0x70 [Tue Mar 17 17:59:55 2020][8592480.392830] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:55 2020][8592480.399879] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:55 2020][8592480.405895] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:55 2020][8592480.413140] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:55 2020][8592480.419853] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.427610] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:56 2020][8592480.433795] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.440592] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.447930] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.455508] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.462676] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.468947] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:56 2020][8592480.475744] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.481974] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.488707] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.495371] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.502424] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:56 2020][8592480.507956] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.515038] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.522814] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:56 2020][8592480.530066] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.537929] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.544917] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:56 2020][8592480.550354] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.556827] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.564413] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:56 2020][8592480.569469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.575737] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:56 2020][8592480.582370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.588657] ll_ost02_058 S ffff9c597d4fd140 0 68717 2 0x00000080 [Tue Mar 17 17:59:56 2020][8592480.595943] Call Trace: [Tue Mar 17 17:59:56 2020][8592480.598572] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:56 2020][8592480.604664] [] schedule+0x29/0x70 [Tue Mar 17 17:59:56 2020][8592480.609849] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.616729] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:56 2020][8592480.622511] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.629003] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.636579] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:56 2020][8592480.641638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.647942] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:56 2020][8592480.654560] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.660834] ll_ost_io01_021 D ffff9c597d4fe180 0 68718 2 0x00000080 [Tue Mar 17 17:59:56 2020][8592480.668120] Call Trace: [Tue Mar 17 17:59:56 2020][8592480.670748] [] schedule+0x29/0x70 [Tue Mar 17 17:59:56 2020][8592480.675893] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.682957] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:56 2020][8592480.688969] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.696212] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.702920] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.710696] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:56 2020][8592480.716884] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.723695] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.731015] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.738602] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.745747] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.752019] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:56 2020][8592480.758830] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.765053] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.771760] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:56 2020][8592480.777830] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.785259] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.792355] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.800104] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:56 2020][8592480.807372] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.815235] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.822200] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:56 2020][8592480.827643] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.834119] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.841692] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:56 2020][8592480.846766] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.853039] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:56 2020][8592480.859658] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592480.865923] ll_ost_io02_023 D ffff9c597d2c8000 0 68719 2 0x00000080 [Tue Mar 17 17:59:56 2020][8592480.873193] Call Trace: [Tue Mar 17 17:59:56 2020][8592480.875837] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.883495] [] schedule+0x29/0x70 [Tue Mar 17 17:59:56 2020][8592480.888667] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.895720] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:56 2020][8592480.901752] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.908989] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.915697] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.923461] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:56 2020][8592480.929643] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592480.936442] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.943772] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.951349] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592480.958489] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.964778] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 17:59:56 2020][8592480.971575] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:56 2020][8592480.977833] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.984568] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.991241] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592480.998294] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 17:59:56 2020][8592481.004344] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.011774] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.018869] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.026623] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:56 2020][8592481.033893] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.041750] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.048748] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:56 2020][8592481.054178] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.060665] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.068230] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:56 2020][8592481.073299] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592481.079566] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:56 2020][8592481.086181] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592481.092470] ll_ost01_062 D ffff9c597d2c9040 0 68720 2 0x00000080 [Tue Mar 17 17:59:56 2020][8592481.099740] Call Trace: [Tue Mar 17 17:59:56 2020][8592481.102369] [] schedule+0x29/0x70 [Tue Mar 17 17:59:56 2020][8592481.107512] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:56 2020][8592481.114593] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:56 2020][8592481.120606] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592481.127847] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:56 2020][8592481.134558] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.142060] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:56 2020][8592481.148245] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592481.155059] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.162376] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.169950] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.177104] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:56 2020][8592481.183379] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:56 2020][8592481.189496] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:56 2020][8592481.196222] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.203999] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.212022] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:56 2020][8592481.218501] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.225594] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.233347] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:56 2020][8592481.240612] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.248476] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.255483] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:56 2020][8592481.260914] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.267399] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:56 2020][8592481.274967] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:56 2020][8592481.280019] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592481.286302] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:56 2020][8592481.292917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:56 2020][8592481.299205] ll_ost_io00_023 D ffff9c597d2ca080 0 68721 2 0x00000080 [Tue Mar 17 17:59:56 2020][8592481.306476] Call Trace: [Tue Mar 17 17:59:56 2020][8592481.309119] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:56 2020][8592481.315736] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:56 2020][8592481.322123] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:56 2020][8592481.327448] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:56 2020][8592481.332945] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.339932] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.347324] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:56 2020][8592481.353340] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:56 2020][8592481.359720] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:56 2020][8592481.366346] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.373079] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:56 2020][8592481.380215] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:56 2020][8592481.385458] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:56 2020][8592481.391123] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.398524] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:56 2020][8592481.405508] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.412825] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:56 2020][8592481.419723] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.427038] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:57 2020][8592481.433920] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:57 2020][8592481.440165] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.446902] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.453567] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.460647] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:57 2020][8592481.467973] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:57 2020][8592481.473720] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:57 2020][8592481.479255] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.486336] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.494085] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:57 2020][8592481.501352] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.509214] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.516207] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:57 2020][8592481.521642] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.528130] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.535696] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:57 2020][8592481.540766] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592481.547034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:57 2020][8592481.553655] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592481.559944] ll_ost_io01_022 D ffff9c597d2cb0c0 0 68722 2 0x00000080 [Tue Mar 17 17:59:57 2020][8592481.567213] Call Trace: [Tue Mar 17 17:59:57 2020][8592481.569842] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:57 2020][8592481.576457] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:57 2020][8592481.582848] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:57 2020][8592481.588166] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:57 2020][8592481.593667] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.600649] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.608047] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:57 2020][8592481.614067] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:57 2020][8592481.620442] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:57 2020][8592481.627062] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:57 2020][8592481.632584] [] ? dqget+0x1a7/0x450 [Tue Mar 17 17:59:57 2020][8592481.637817] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:57 2020][8592481.644950] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:57 2020][8592481.650198] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:57 2020][8592481.655865] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.663265] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:57 2020][8592481.670253] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.677570] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.684463] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.691781] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:57 2020][8592481.698688] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:57 2020][8592481.704904] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.711650] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.718301] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.725370] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:57 2020][8592481.732697] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:57 2020][8592481.738447] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:57 2020][8592481.743986] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.751070] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.758819] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:57 2020][8592481.766100] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.773957] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.780935] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:57 2020][8592481.786368] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.792855] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.800423] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:57 2020][8592481.805499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592481.811767] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:57 2020][8592481.818390] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592481.824678] ll_ost_io00_024 D ffff9c597d2cc100 0 68723 2 0x00000080 [Tue Mar 17 17:59:57 2020][8592481.831965] Call Trace: [Tue Mar 17 17:59:57 2020][8592481.834597] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:57 2020][8592481.841217] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:57 2020][8592481.847591] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:57 2020][8592481.852907] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:57 2020][8592481.858403] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.865384] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.872782] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:57 2020][8592481.878813] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:57 2020][8592481.885180] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:57 2020][8592481.891801] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.898530] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:57 2020][8592481.905670] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:57 2020][8592481.910896] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:57 2020][8592481.916578] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.923980] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:57 2020][8592481.930948] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.938277] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.945161] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592481.952474] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:57 2020][8592481.959384] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:57 2020][8592481.965599] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.972319] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:57 2020][8592481.977845] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.984928] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592481.992693] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:57 2020][8592481.999945] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.007801] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.014782] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:57 2020][8592482.020213] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.026715] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.034286] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:57 2020][8592482.039360] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.045630] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:57 2020][8592482.052251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.058532] ll_ost00_054 S ffff9c597d2cd140 0 68724 2 0x00000080 [Tue Mar 17 17:59:57 2020][8592482.065804] Call Trace: [Tue Mar 17 17:59:57 2020][8592482.068436] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:57 2020][8592482.074532] [] schedule+0x29/0x70 [Tue Mar 17 17:59:57 2020][8592482.079715] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.086604] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:57 2020][8592482.092381] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.098869] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.106436] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:57 2020][8592482.111514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.117780] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:57 2020][8592482.124407] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.130677] ll_ost02_059 D ffff9c597d2ce180 0 68725 2 0x00000080 [Tue Mar 17 17:59:57 2020][8592482.137953] Call Trace: [Tue Mar 17 17:59:57 2020][8592482.140597] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:57 2020][8592482.145916] [] schedule+0x29/0x70 [Tue Mar 17 17:59:57 2020][8592482.151087] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 17:59:57 2020][8592482.157133] [] ? at_measured+0x108/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.163666] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:57 2020][8592482.170905] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:57 2020][8592482.176656] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 17:59:57 2020][8592482.182494] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:57 2020][8592482.187821] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 17:59:57 2020][8592482.194004] [] io_schedule+0x18/0x20 [Tue Mar 17 17:59:57 2020][8592482.199403] [] bit_wait_io+0x11/0x50 [Tue Mar 17 17:59:57 2020][8592482.204829] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 17:59:57 2020][8592482.210844] [] ? bit_wait+0x50/0x50 [Tue Mar 17 17:59:57 2020][8592482.216179] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 17:59:57 2020][8592482.223054] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.229151] [] __lock_buffer+0x32/0x40 [Tue Mar 17 17:59:57 2020][8592482.234747] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 17:59:57 2020][8592482.241629] [] ? __getblk+0x2d/0x300 [Tue Mar 17 17:59:57 2020][8592482.247039] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 17:59:57 2020][8592482.254651] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.262925] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.270614] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:57 2020][8592482.277597] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.284741] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.292244] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.299216] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 17:59:57 2020][8592482.306008] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 17:59:57 2020][8592482.312122] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:57 2020][8592482.318849] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.326622] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.334641] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:57 2020][8592482.341120] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.348229] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.355983] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:57 2020][8592482.363249] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.371111] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.378092] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:57 2020][8592482.383522] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.389997] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:57 2020][8592482.397583] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:57 2020][8592482.402639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.408945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:57 2020][8592482.415559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:57 2020][8592482.421826] ll_ost00_055 S ffff9c7a3693e180 0 68726 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.429112] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.431740] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592482.437835] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.443026] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.449906] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592482.455684] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.462170] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.469743] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592482.474825] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.481099] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592482.487718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.494000] ll_ost01_063 S ffff9c7a3693c100 0 68727 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.501271] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.503901] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592482.510003] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.515194] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.522075] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592482.527858] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.534348] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.541920] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592482.546987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.553258] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592482.559877] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.566161] ll_ost00_056 S ffff9c597dbc8000 0 68728 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.573430] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.576061] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592482.582176] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.587360] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.594234] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592482.600031] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.606502] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.614086] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592482.619142] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.625429] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592482.632047] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.638320] ll_ost00_057 S ffff9c597dbc9040 0 68729 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.645607] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.648236] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592482.654338] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.659542] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.666420] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592482.672203] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.678691] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.686266] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592482.691344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.697611] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592482.704233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.710535] ll_ost_io00_025 D ffff9c597dbca080 0 68730 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.717808] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.720438] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.727052] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:58 2020][8592482.733426] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:58 2020][8592482.738744] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:58 2020][8592482.744260] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.751228] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.758626] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:58 2020][8592482.764660] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:58 2020][8592482.771038] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:58 2020][8592482.777655] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:58 2020][8592482.783157] [] ? dqget+0x1a7/0x450 [Tue Mar 17 17:59:58 2020][8592482.788387] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:58 2020][8592482.795526] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:58 2020][8592482.800769] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:58 2020][8592482.806434] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.813835] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:58 2020][8592482.820821] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.828137] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.835032] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:58 2020][8592482.842349] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:58 2020][8592482.849243] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:58 2020][8592482.855458] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.862195] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.868857] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.875933] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 17:59:58 2020][8592482.883255] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 17:59:58 2020][8592482.889006] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:58 2020][8592482.894527] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.901633] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.909378] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:58 2020][8592482.916644] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.924499] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.931465] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:58 2020][8592482.936919] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.943393] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592482.950979] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592482.956036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.962324] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592482.968954] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592482.975238] ll_ost01_064 S ffff9c597dbcb0c0 0 68731 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592482.982508] Call Trace: [Tue Mar 17 17:59:58 2020][8592482.985139] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592482.991239] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592482.996423] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.003305] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.009104] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.015580] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.023152] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.028240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.034512] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.041135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.047422] ll_ost02_060 S ffff9c597dbcc100 0 68732 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.054694] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.057325] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.063426] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592483.068615] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.075488] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.081271] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.087775] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.095346] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.100406] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.106695] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.113312] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.119585] ll_ost00_058 S ffff9c597dbcd140 0 68733 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.126878] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.129508] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.135602] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592483.140791] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.147681] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.153455] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.159953] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.167522] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.172581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.178861] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.185477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.191745] ll_ost01_065 S ffff9c597dbce180 0 68734 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.199038] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.201668] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.207768] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592483.212968] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.219843] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.225626] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.232114] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.239689] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.244752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.251038] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.257670] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.263952] ll_ost03_061 S ffff9c597d708000 0 68735 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.271224] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.273851] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.279975] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592483.285148] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.292027] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.297824] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.304292] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.311865] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.316935] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.323211] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.329831] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.336120] ll_ost01_066 S ffff9c597d709040 0 68736 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.343391] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.346021] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.352122] [] schedule+0x29/0x70 [Tue Mar 17 17:59:58 2020][8592483.357304] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.364185] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:58 2020][8592483.369963] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.376449] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:58 2020][8592483.384025] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:58 2020][8592483.389112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.395385] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:58 2020][8592483.401999] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:58 2020][8592483.408280] ll_ost02_061 S ffff9c597d70a080 0 68737 2 0x00000080 [Tue Mar 17 17:59:58 2020][8592483.415551] Call Trace: [Tue Mar 17 17:59:58 2020][8592483.418180] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:58 2020][8592483.424282] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.429471] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.436346] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.442128] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.448632] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.456201] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.461262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.467552] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.474167] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.480441] ll_ost00_059 S ffff9c597d70b0c0 0 68738 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.487726] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.490357] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.496459] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.501649] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.508538] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.514314] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.520802] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.528369] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.533432] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.539718] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.546335] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.552608] ll_ost01_067 S ffff9c597d70c100 0 68739 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.559895] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.562523] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.568617] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.573821] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.580699] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.586480] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.592970] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.600536] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.605589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.611880] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.618493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.624759] ll_ost02_062 S ffff9c597d70d140 0 68740 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.632053] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.634684] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.640800] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.645971] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.652849] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.658645] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.665116] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.672688] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.677762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.684033] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.690653] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.696955] ll_ost01_068 S ffff9c597d70e180 0 68741 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.704230] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.706860] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.712953] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.718143] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.725017] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.730800] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.737287] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.744854] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.749917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.756220] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.762837] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.769104] ll_ost00_060 S ffff9c597d3f8000 0 68742 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.776398] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.779028] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.785128] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.790314] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.797192] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.802969] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.809470] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.817041] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.822117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.828384] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.834996] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.841278] ll_ost02_063 S ffff9c597d3f9040 0 68743 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.848549] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.851178] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592483.857280] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.862469] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.869357] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592483.875135] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.881623] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592483.889189] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592483.894241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.900525] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592483.907139] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592483.913412] ll_ost03_062 D ffff9c597d3fa080 0 68744 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592483.920699] Call Trace: [Tue Mar 17 17:59:59 2020][8592483.923329] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592483.928481] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 17:59:59 2020][8592483.935558] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 17:59:59 2020][8592483.941575] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 17:59:59 2020][8592483.948817] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 17:59:59 2020][8592483.955527] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592483.963030] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 17:59:59 2020][8592483.969213] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:59 2020][8592483.976009] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592483.983341] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 17:59:59 2020][8592483.990919] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592483.998079] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 17:59:59 2020][8592484.004371] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 17:59:59 2020][8592484.010467] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 17:59:59 2020][8592484.017212] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.024979] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.033000] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 17:59:59 2020][8592484.039480] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.046567] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.054326] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:59 2020][8592484.061575] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.069463] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.076427] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:59 2020][8592484.081864] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.088351] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.095923] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592484.100980] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592484.107267] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592484.113885] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592484.120158] ll_ost02_064 S ffff9c597d3fb0c0 0 68745 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592484.127445] Call Trace: [Tue Mar 17 17:59:59 2020][8592484.130074] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 17:59:59 2020][8592484.136173] [] schedule+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592484.141372] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.148247] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 17:59:59 2020][8592484.154032] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.160519] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.168088] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592484.173163] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592484.179431] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592484.186066] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592484.192350] ll_ost_io01_023 D ffff9c597d3fc100 0 68746 2 0x00000080 [Tue Mar 17 17:59:59 2020][8592484.199622] Call Trace: [Tue Mar 17 17:59:59 2020][8592484.202264] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 17:59:59 2020][8592484.208880] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 17:59:59 2020][8592484.215256] [] mutex_lock+0x1f/0x2f [Tue Mar 17 17:59:59 2020][8592484.220571] [] dquot_commit+0x29/0xd0 [Tue Mar 17 17:59:59 2020][8592484.226066] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.233045] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.240444] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 17:59:59 2020][8592484.246466] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 17:59:59 2020][8592484.252852] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 17:59:59 2020][8592484.259485] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 17:59:59 2020][8592484.266631] [] ? dqget+0x190/0x450 [Tue Mar 17 17:59:59 2020][8592484.271862] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 17:59:59 2020][8592484.277548] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.284966] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 17:59:59 2020][8592484.291939] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.299270] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.306152] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 17:59:59 2020][8592484.313466] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 17:59:59 2020][8592484.320376] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 17:59:59 2020][8592484.326592] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.333329] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.339992] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.347045] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 17:59:59 2020][8592484.352589] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.359669] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.367416] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 17:59:59 2020][8592484.374682] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.382538] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.389531] [] ? __wake_up+0x44/0x50 [Tue Mar 17 17:59:59 2020][8592484.394966] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.401455] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 17:59:59 2020][8592484.409028] [] kthread+0xd1/0xe0 [Tue Mar 17 17:59:59 2020][8592484.414097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 17:59:59 2020][8592484.420368] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 17:59:59 2020][8592484.426987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.433267] ll_ost_io02_024 D ffff9c597d3fd140 0 68747 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592484.440538] Call Trace: [Tue Mar 17 18:00:00 2020][8592484.443183] [] schedule+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592484.448328] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:00 2020][8592484.455407] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:00 2020][8592484.461422] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:00 2020][8592484.468665] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:00 2020][8592484.474500] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:00 2020][8592484.481215] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.488529] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:00 2020][8592484.494734] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:00 2020][8592484.501527] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.508842] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.516445] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.523590] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:00 2020][8592484.530386] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:00 2020][8592484.536616] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.543351] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.550012] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.557068] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:00 2020][8592484.562608] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.569691] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.577450] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:00 2020][8592484.584707] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.592597] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.599558] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:00 2020][8592484.604988] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.611475] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.619040] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592484.624108] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.630379] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592484.636999] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.643288] ll_ost_io03_022 D ffff9c597d3fe180 0 68748 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592484.650576] Call Trace: [Tue Mar 17 18:00:00 2020][8592484.653206] [] schedule+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592484.658359] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.665430] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.672567] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:00 2020][8592484.678576] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:00 2020][8592484.684780] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:00 2020][8592484.691580] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:00 2020][8592484.697807] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.704543] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.711217] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.718290] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:00 2020][8592484.725613] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:00 2020][8592484.731363] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:00 2020][8592484.736889] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.743984] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.751732] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:00 2020][8592484.758986] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.766861] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.773821] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:00 2020][8592484.779258] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.785761] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.793329] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592484.798381] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.804664] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592484.811277] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.817545] ll_ost03_063 S ffff9c597d590000 0 68749 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592484.824838] Call Trace: [Tue Mar 17 18:00:00 2020][8592484.827467] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:00 2020][8592484.833561] [] schedule+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592484.838765] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.845642] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:00 2020][8592484.851427] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.857913] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592484.865479] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592484.870546] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.876816] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592484.883428] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592484.889707] ll_ost_io00_026 D ffff9c597d591040 0 68750 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592484.896979] Call Trace: [Tue Mar 17 18:00:00 2020][8592484.899609] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592484.906231] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:00 2020][8592484.912610] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:00 2020][8592484.917932] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:00 2020][8592484.923433] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.930414] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.937813] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:00 2020][8592484.943831] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:00 2020][8592484.950210] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:00 2020][8592484.956831] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.963536] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:00 2020][8592484.970690] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:00 2020][8592484.975923] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:00 2020][8592484.981610] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592484.989010] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:00 2020][8592484.995981] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.003310] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.010192] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.017507] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:00 2020][8592485.024402] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:00 2020][8592485.030637] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.037372] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.044033] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.051087] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:00 2020][8592485.056627] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.063710] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.071458] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:00 2020][8592485.078711] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.086571] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.093537] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:00 2020][8592485.098988] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.105464] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.113035] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592485.118088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.124353] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592485.130967] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.137232] ll_ost01_069 S ffff9c597d592080 0 68751 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592485.144501] Call Trace: [Tue Mar 17 18:00:00 2020][8592485.147132] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:00 2020][8592485.153232] [] schedule+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592485.158415] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.165287] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:00 2020][8592485.171062] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.177539] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.185108] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592485.190161] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.196427] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592485.203041] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.209317] ll_ost_io01_024 D ffff9c597d5930c0 0 68752 2 0x00000080 [Tue Mar 17 18:00:00 2020][8592485.216609] Call Trace: [Tue Mar 17 18:00:00 2020][8592485.219237] [] schedule+0x29/0x70 [Tue Mar 17 18:00:00 2020][8592485.224381] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:00 2020][8592485.231431] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:00 2020][8592485.237440] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:00 2020][8592485.244667] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:00 2020][8592485.250509] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:00 2020][8592485.257240] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.264549] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:00 2020][8592485.270738] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:00 2020][8592485.277552] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.284870] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.292445] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:00 2020][8592485.299584] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:00 2020][8592485.306378] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:00 2020][8592485.312595] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.319333] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.325982] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.333032] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:00 2020][8592485.339064] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.346505] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.353587] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.361335] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:00 2020][8592485.368586] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.376446] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.383412] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:00 2020][8592485.388848] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.395324] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:00 2020][8592485.402894] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:00 2020][8592485.407947] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.414236] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:00 2020][8592485.420852] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:00 2020][8592485.427118] ll_ost03_064 S ffff9c597d594100 0 68753 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592485.434386] Call Trace: [Tue Mar 17 18:00:01 2020][8592485.437015] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:01 2020][8592485.443108] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592485.448277] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.455154] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:01 2020][8592485.460931] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.467404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.474991] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592485.480047] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.486320] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592485.492933] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.499200] ll_ost_io02_025 D ffff9c597d595140 0 68754 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592485.506476] Call Trace: [Tue Mar 17 18:00:01 2020][8592485.509106] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592485.514258] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.521309] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:01 2020][8592485.527324] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.534569] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.541281] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.549025] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:01 2020][8592485.555216] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.562012] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.569329] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.576900] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.584043] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:01 2020][8592485.590316] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:01 2020][8592485.596852] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:01 2020][8592485.603170] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.610252] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.617999] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:01 2020][8592485.625253] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.633111] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.640078] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:01 2020][8592485.645512] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.651986] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.659560] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592485.664623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.670910] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592485.677525] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.683791] ll_ost02_065 S ffff9c597d596180 0 68755 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592485.691060] Call Trace: [Tue Mar 17 18:00:01 2020][8592485.693689] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:01 2020][8592485.699783] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592485.704960] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.711839] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:01 2020][8592485.717621] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.724097] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.731681] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592485.736736] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.743004] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592485.749623] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.755889] ll_ost_io00_027 D ffff9c597db90000 0 68756 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592485.763158] Call Trace: [Tue Mar 17 18:00:01 2020][8592485.765786] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592485.770936] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.777997] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.785131] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:01 2020][8592485.791157] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:01 2020][8592485.797346] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:01 2020][8592485.804142] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:01 2020][8592485.810359] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.817060] [] ? ___slab_alloc+0x365/0x4f0 [Tue Mar 17 18:00:01 2020][8592485.822981] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:01 2020][8592485.829019] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.836446] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.843525] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.851273] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:01 2020][8592485.858526] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.866401] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.873369] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:01 2020][8592485.878804] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.885279] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592485.892851] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592485.897903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.904171] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592485.910790] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592485.917057] ll_ost_io03_023 D ffff9c597db91040 0 68757 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592485.924350] Call Trace: [Tue Mar 17 18:00:01 2020][8592485.926981] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592485.932133] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.939184] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:01 2020][8592485.945199] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.952427] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:01 2020][8592485.958271] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.964984] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.972298] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:01 2020][8592485.978480] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:01 2020][8592485.985295] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592485.992613] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.000185] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.007325] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:01 2020][8592486.014119] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:01 2020][8592486.020336] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.027069] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.033713] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.040765] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:01 2020][8592486.046796] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.054236] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.061318] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.069066] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:01 2020][8592486.076319] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.084181] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.091147] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:01 2020][8592486.096582] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.103056] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.110627] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592486.115681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592486.121972] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592486.128589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592486.134876] ll_ost02_066 S ffff9c597db92080 0 68758 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592486.142164] Call Trace: [Tue Mar 17 18:00:01 2020][8592486.144794] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:01 2020][8592486.150913] [] schedule+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592486.156116] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.162996] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:01 2020][8592486.168794] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.175271] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.182842] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:01 2020][8592486.187903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592486.194176] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:01 2020][8592486.200811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:01 2020][8592486.207080] ll_ost_io02_026 D ffff9c597db930c0 0 68759 2 0x00000080 [Tue Mar 17 18:00:01 2020][8592486.214367] Call Trace: [Tue Mar 17 18:00:01 2020][8592486.216998] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:01 2020][8592486.223618] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:01 2020][8592486.229980] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:01 2020][8592486.235302] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:01 2020][8592486.240804] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.247780] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.255183] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:01 2020][8592486.261204] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:01 2020][8592486.267574] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:01 2020][8592486.274202] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.280907] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:01 2020][8592486.288046] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:01 2020][8592486.293279] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:01 2020][8592486.298950] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.306363] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:01 2020][8592486.313332] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.320646] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.327528] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:01 2020][8592486.334843] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:01 2020][8592486.341722] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:01 2020][8592486.347950] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.354687] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.361335] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.368388] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:01 2020][8592486.374442] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.381870] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.388963] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.396717] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:01 2020][8592486.403975] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.411835] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.418804] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:01 2020][8592486.424239] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:01 2020][8592486.430712] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.438300] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:02 2020][8592486.443353] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.449621] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:02 2020][8592486.456242] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.462515] ll_ost_io01_025 D ffff9c597db94100 0 68760 2 0x00000080 [Tue Mar 17 18:00:02 2020][8592486.469784] Call Trace: [Tue Mar 17 18:00:02 2020][8592486.472423] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.480084] [] schedule+0x29/0x70 [Tue Mar 17 18:00:02 2020][8592486.485235] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.492288] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:02 2020][8592486.498316] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.505541] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.512251] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.519998] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:02 2020][8592486.526184] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.532981] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.540299] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.547881] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.555030] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:02 2020][8592486.561302] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:02 2020][8592486.568112] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:02 2020][8592486.574333] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.581072] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.587720] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.594798] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:02 2020][8592486.602112] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:02 2020][8592486.607866] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:02 2020][8592486.613393] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.620481] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.628229] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:02 2020][8592486.635495] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.643359] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.650327] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:02 2020][8592486.655759] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.662235] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.669806] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:02 2020][8592486.674869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.681143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:02 2020][8592486.687763] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.694039] ll_ost00_061 S ffff9c597db95140 0 68761 2 0x00000080 [Tue Mar 17 18:00:02 2020][8592486.701333] Call Trace: [Tue Mar 17 18:00:02 2020][8592486.703962] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:02 2020][8592486.710064] [] schedule+0x29/0x70 [Tue Mar 17 18:00:02 2020][8592486.715240] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.722121] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:02 2020][8592486.728740] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:02 2020][8592486.734523] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.740997] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592486.748571] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:02 2020][8592486.753632] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.759920] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:02 2020][8592486.766536] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.772811] ll_ost_io03_024 D ffff9c597db96180 0 68762 2 0x00000080 [Tue Mar 17 18:00:02 2020][8592486.780088] Call Trace: [Tue Mar 17 18:00:02 2020][8592486.782716] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:02 2020][8592486.788037] [] schedule+0x29/0x70 [Tue Mar 17 18:00:02 2020][8592486.793185] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:02 2020][8592486.799192] [] ? wake_up_bit+0x25/0x30 [Tue Mar 17 18:00:02 2020][8592486.804779] [] ? do_get_write_access+0x357/0x4d0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.811831] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:02 2020][8592486.817683] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:02 2020][8592486.823000] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:02 2020][8592486.829189] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:02 2020][8592486.834595] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:02 2020][8592486.840003] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:00:02 2020][8592486.846018] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:02 2020][8592486.851338] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:00:02 2020][8592486.858214] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592486.864317] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:00:02 2020][8592486.869899] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.876792] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:00:02 2020][8592486.882193] [] ? internal_add_timer+0x7d/0xc0 [Tue Mar 17 18:00:02 2020][8592486.888386] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.895964] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.904241] [] ldiskfs_mb_mark_diskspace_used+0x7d/0x510 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.912258] [] ldiskfs_mb_new_blocks+0x350/0xb20 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.919582] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.927244] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:02 2020][8592486.932833] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.940501] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.948446] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.955848] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.962907] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.971092] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:02 2020][8592486.978064] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.985299] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592486.992612] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:02 2020][8592486.999405] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:02 2020][8592487.005624] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.012363] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.019026] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.026080] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:02 2020][8592487.032119] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.039554] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.046643] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.054389] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:02 2020][8592487.061640] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.069503] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.076467] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:02 2020][8592487.081904] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.088393] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.095966] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:02 2020][8592487.101028] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592487.107304] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:02 2020][8592487.113928] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592487.120198] ll_ost02_067 S ffff9c597e030000 0 68763 2 0x00000080 [Tue Mar 17 18:00:02 2020][8592487.127468] Call Trace: [Tue Mar 17 18:00:02 2020][8592487.130096] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:02 2020][8592487.136199] [] schedule+0x29/0x70 [Tue Mar 17 18:00:02 2020][8592487.141374] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.148269] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:02 2020][8592487.154045] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.160519] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.168092] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:02 2020][8592487.173152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592487.179430] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:02 2020][8592487.186048] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:02 2020][8592487.192324] ll_ost_io00_028 D ffff9c597e031040 0 68764 2 0x00000080 [Tue Mar 17 18:00:02 2020][8592487.199600] Call Trace: [Tue Mar 17 18:00:02 2020][8592487.202230] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:02 2020][8592487.208851] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:02 2020][8592487.215229] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:02 2020][8592487.220553] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:02 2020][8592487.226053] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.233029] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.240432] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:02 2020][8592487.246715] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.253604] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.261985] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.269213] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.276878] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:02 2020][8592487.282481] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.290142] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.298064] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.305464] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.312523] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.320707] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:02 2020][8592487.327694] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.334924] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:02 2020][8592487.342242] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:02 2020][8592487.349055] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:02 2020][8592487.355278] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.362012] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.368662] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.375713] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:02 2020][8592487.381751] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.389186] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.396274] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.404022] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:02 2020][8592487.411275] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.419151] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:02 2020][8592487.426118] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:02 2020][8592487.431556] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.438028] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.445602] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592487.450660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.456932] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592487.463548] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.469823] ll_ost03_065 S ffff9c597e032080 0 68765 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592487.477108] Call Trace: [Tue Mar 17 18:00:03 2020][8592487.479737] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:03 2020][8592487.485832] [] schedule+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592487.491023] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.497903] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:03 2020][8592487.503687] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.510164] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.517735] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592487.522795] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.529069] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592487.535691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.541978] ll_ost_io02_027 D ffff9c597e0330c0 0 68766 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592487.549250] Call Trace: [Tue Mar 17 18:00:03 2020][8592487.551879] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592487.558502] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:03 2020][8592487.564864] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:03 2020][8592487.570184] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:03 2020][8592487.575687] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.582661] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.590059] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:03 2020][8592487.596338] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.603242] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.611600] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.618837] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.626500] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:03 2020][8592487.632088] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.639749] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.647673] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.655079] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.662138] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.670324] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:03 2020][8592487.677295] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.684529] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.691843] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:03 2020][8592487.698638] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:03 2020][8592487.704858] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.711595] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.718242] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.725317] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:03 2020][8592487.732635] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:03 2020][8592487.738389] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:03 2020][8592487.743934] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.751018] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.758767] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:03 2020][8592487.766023] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.773881] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.780846] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:03 2020][8592487.786283] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.792758] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.800328] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592487.805391] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.811681] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592487.818295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.824570] ll_ost01_070 D ffff9c597e034100 0 68767 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592487.831846] Call Trace: [Tue Mar 17 18:00:03 2020][8592487.834478] [] schedule+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592487.839629] [] jbd2_log_wait_commit+0xc5/0x140 [jbd2] [Tue Mar 17 18:00:03 2020][8592487.846506] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:03 2020][8592487.852522] [] jbd2_complete_transaction+0x52/0xa0 [jbd2] [Tue Mar 17 18:00:03 2020][8592487.859754] [] ldiskfs_sync_file+0x292/0x320 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.866719] [] vfs_fsync_range+0x20/0x30 [Tue Mar 17 18:00:03 2020][8592487.872491] [] osd_object_sync+0xb1/0x160 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592487.879575] [] tgt_sync+0xb7/0x270 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.885587] [] ofd_sync_hdl+0x111/0x530 [ofd] [Tue Mar 17 18:00:03 2020][8592487.891802] [] ? lustre_pack_reply+0x11/0x20 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.898712] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.905801] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.913558] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:03 2020][8592487.920809] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.928668] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.935633] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:03 2020][8592487.941086] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.947563] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592487.955135] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592487.960196] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.966472] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592487.973091] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592487.979364] ll_ost_io01_026 D ffff9c597e035140 0 68768 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592487.986652] Call Trace: [Tue Mar 17 18:00:03 2020][8592487.989281] [] schedule+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592487.994432] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.001498] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:03 2020][8592488.007510] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.014737] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:03 2020][8592488.020579] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.027294] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.034608] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:03 2020][8592488.040800] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.047596] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.054913] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.062496] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.069659] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:03 2020][8592488.076456] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:03 2020][8592488.082671] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.089404] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.096051] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.103103] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:03 2020][8592488.108628] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.115708] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.123455] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:03 2020][8592488.130707] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.138583] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.145551] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:03 2020][8592488.150989] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.157461] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.165033] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592488.170097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592488.176371] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592488.182991] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592488.189265] ll_ost00_062 S ffff9c597e036180 0 68769 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592488.196532] Call Trace: [Tue Mar 17 18:00:03 2020][8592488.199164] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:03 2020][8592488.205266] [] schedule+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592488.210458] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.217338] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:03 2020][8592488.223123] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.229596] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.237167] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:03 2020][8592488.242229] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592488.248504] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:03 2020][8592488.255125] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:03 2020][8592488.261412] ll_ost_io00_029 D ffff9c597ec98000 0 68770 2 0x00000080 [Tue Mar 17 18:00:03 2020][8592488.268702] Call Trace: [Tue Mar 17 18:00:03 2020][8592488.271339] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.279002] [] schedule+0x29/0x70 [Tue Mar 17 18:00:03 2020][8592488.284153] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.291204] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:03 2020][8592488.297220] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.304451] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.311174] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.318921] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:03 2020][8592488.325110] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:03 2020][8592488.331922] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.339241] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.346817] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:03 2020][8592488.353963] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:03 2020][8592488.360239] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:03 2020][8592488.367033] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:03 2020][8592488.373249] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.379989] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.386638] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.393704] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:03 2020][8592488.399233] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.406321] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:03 2020][8592488.414067] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:04 2020][8592488.421321] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.429183] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.436146] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:04 2020][8592488.441584] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.448057] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.455630] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:04 2020][8592488.460704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.466973] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:04 2020][8592488.473596] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.479870] ll_ost02_068 S ffff9c597ec99040 0 68771 2 0x00000080 [Tue Mar 17 18:00:04 2020][8592488.487147] Call Trace: [Tue Mar 17 18:00:04 2020][8592488.489776] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:04 2020][8592488.495878] [] schedule+0x29/0x70 [Tue Mar 17 18:00:04 2020][8592488.501056] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.507935] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:04 2020][8592488.513715] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.520207] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.527778] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:04 2020][8592488.532833] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.539108] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:04 2020][8592488.545728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.552002] ll_ost_io03_025 D ffff9c597ec9a080 0 68772 2 0x00000080 [Tue Mar 17 18:00:04 2020][8592488.559278] Call Trace: [Tue Mar 17 18:00:04 2020][8592488.561906] [] schedule+0x29/0x70 [Tue Mar 17 18:00:04 2020][8592488.567053] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:04 2020][8592488.574103] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:04 2020][8592488.580134] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592488.587366] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:04 2020][8592488.594079] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.601394] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:04 2020][8592488.607585] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592488.614383] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.621702] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.629281] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.636422] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:04 2020][8592488.643217] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:04 2020][8592488.649447] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.656181] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.662828] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.669878] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:04 2020][8592488.675920] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.683353] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.690433] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.698181] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:04 2020][8592488.705435] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.713294] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.720274] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:04 2020][8592488.725704] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.732178] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.739750] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:04 2020][8592488.744812] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.751089] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:04 2020][8592488.757707] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592488.763982] ll_ost_io03_026 D ffff9c597ec9b0c0 0 68773 2 0x00000080 [Tue Mar 17 18:00:04 2020][8592488.771258] Call Trace: [Tue Mar 17 18:00:04 2020][8592488.773888] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:04 2020][8592488.780510] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:04 2020][8592488.786885] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:04 2020][8592488.792200] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:04 2020][8592488.797695] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.804671] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.812075] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:04 2020][8592488.818351] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:00:04 2020][8592488.823504] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.830738] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.837882] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 18:00:04 2020][8592488.844780] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:04 2020][8592488.850361] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.858024] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.865694] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:00:04 2020][8592488.872237] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.879647] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.886710] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.894898] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592488.901868] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.909105] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592488.916441] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:04 2020][8592488.923236] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:04 2020][8592488.929455] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.936194] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.942844] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.949923] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:04 2020][8592488.957233] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:04 2020][8592488.962989] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:04 2020][8592488.968515] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.975605] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.983375] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:04 2020][8592488.990629] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592488.998491] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.005455] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:04 2020][8592489.010892] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.017365] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.024940] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:04 2020][8592489.029999] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592489.036273] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:04 2020][8592489.042894] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592489.049184] ll_ost_io02_028 D ffff9c597ec9c100 0 68774 2 0x00000080 [Tue Mar 17 18:00:04 2020][8592489.056454] Call Trace: [Tue Mar 17 18:00:04 2020][8592489.059086] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:04 2020][8592489.065707] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:04 2020][8592489.072067] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:04 2020][8592489.077389] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:04 2020][8592489.082889] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.089868] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.097270] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:04 2020][8592489.103291] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:04 2020][8592489.109676] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:04 2020][8592489.116292] [] ? try_wait_for_completion+0x20/0x50 [Tue Mar 17 18:00:04 2020][8592489.122910] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:04 2020][8592489.130047] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:04 2020][8592489.135283] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:04 2020][8592489.140959] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.148363] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592489.155335] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.162656] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.169538] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.176866] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:04 2020][8592489.183750] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:04 2020][8592489.189970] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.196681] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:00:04 2020][8592489.202346] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:04 2020][8592489.208101] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:04 2020][8592489.214663] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.222097] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.229183] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.236934] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:04 2020][8592489.244204] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.252061] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.259026] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:04 2020][8592489.264480] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.270957] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:04 2020][8592489.278532] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:04 2020][8592489.283587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592489.289853] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:04 2020][8592489.296474] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:04 2020][8592489.302750] ll_ost_io02_029 D ffff9c597ec9d140 0 68775 2 0x00000080 [Tue Mar 17 18:00:04 2020][8592489.310042] Call Trace: [Tue Mar 17 18:00:04 2020][8592489.312680] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.320343] [] schedule+0x29/0x70 [Tue Mar 17 18:00:04 2020][8592489.325495] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:04 2020][8592489.332546] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:04 2020][8592489.338564] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592489.345790] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:04 2020][8592489.352502] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.360247] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:04 2020][8592489.366436] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:04 2020][8592489.373247] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.380567] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.388148] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:04 2020][8592489.395288] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:04 2020][8592489.401564] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:04 2020][8592489.408357] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:04 2020][8592489.414574] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.421329] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.427979] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.435050] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:05 2020][8592489.442377] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:05 2020][8592489.448124] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:05 2020][8592489.453645] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.460731] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.468480] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:05 2020][8592489.475732] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.483591] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.490558] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:05 2020][8592489.495994] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.502469] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.510054] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592489.515111] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.521386] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592489.528008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.534282] ll_ost00_063 S ffff9c597ec9e180 0 68776 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592489.541558] Call Trace: [Tue Mar 17 18:00:05 2020][8592489.544188] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:05 2020][8592489.550288] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592489.555466] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.562344] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:05 2020][8592489.568980] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:05 2020][8592489.574758] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.581232] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.588804] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592489.593865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.600141] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592489.606761] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.613037] ll_ost_io02_030 D ffff9c4b6968a080 0 68777 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592489.620310] Call Trace: [Tue Mar 17 18:00:05 2020][8592489.622942] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592489.628094] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:05 2020][8592489.635160] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:05 2020][8592489.641169] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:05 2020][8592489.648400] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:05 2020][8592489.655110] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592489.662853] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:05 2020][8592489.669044] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:05 2020][8592489.675839] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592489.683157] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:05 2020][8592489.690740] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592489.697901] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:05 2020][8592489.704178] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:05 2020][8592489.710974] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:05 2020][8592489.717191] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.723903] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:05 2020][8592489.729858] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.736507] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.743556] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:05 2020][8592489.749597] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.757030] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.764124] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.771874] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:05 2020][8592489.779128] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.786989] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.793954] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:05 2020][8592489.799391] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.805864] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.813437] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592489.818497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.824773] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592489.831407] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.837678] ll_ost02_069 S ffff9c599174b0c0 0 68778 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592489.844952] Call Trace: [Tue Mar 17 18:00:05 2020][8592489.847581] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:05 2020][8592489.853676] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592489.858854] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.865733] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:05 2020][8592489.871516] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.877990] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.885560] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592489.890639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.896914] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592489.903535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.909811] ll_ost01_071 S ffff9c5991748000 0 68779 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592489.917085] Call Trace: [Tue Mar 17 18:00:05 2020][8592489.919717] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:05 2020][8592489.925819] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592489.930995] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.937874] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:05 2020][8592489.943656] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.950145] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592489.957720] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592489.962782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.969056] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592489.975678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592489.981953] ll_ost_io03_027 D ffff9c599174c100 0 68780 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592489.989238] Call Trace: [Tue Mar 17 18:00:05 2020][8592489.991867] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592489.997020] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.004071] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:05 2020][8592490.010104] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:05 2020][8592490.016293] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:05 2020][8592490.023088] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:05 2020][8592490.029306] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.036045] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.042692] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.049762] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:05 2020][8592490.057078] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:05 2020][8592490.062833] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:05 2020][8592490.068358] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.075459] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.083210] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:05 2020][8592490.090462] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.098324] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.105290] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:05 2020][8592490.110727] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.117199] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.124771] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592490.129833] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592490.136108] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592490.142744] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592490.149012] ll_ost02_070 S ffff9c599174a080 0 68781 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592490.156288] Call Trace: [Tue Mar 17 18:00:05 2020][8592490.158917] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:05 2020][8592490.165012] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592490.170190] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.177068] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:05 2020][8592490.182848] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.189327] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.196896] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:05 2020][8592490.201957] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592490.208248] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:05 2020][8592490.214861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:05 2020][8592490.221136] ll_ost_io01_027 D ffff9c599174e180 0 68782 2 0x00000080 [Tue Mar 17 18:00:05 2020][8592490.228414] Call Trace: [Tue Mar 17 18:00:05 2020][8592490.231052] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.238713] [] schedule+0x29/0x70 [Tue Mar 17 18:00:05 2020][8592490.243863] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:05 2020][8592490.250915] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:05 2020][8592490.256934] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:05 2020][8592490.264183] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:05 2020][8592490.270905] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.278649] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:05 2020][8592490.284839] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:05 2020][8592490.291637] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.298954] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.306537] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:05 2020][8592490.313683] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:05 2020][8592490.319960] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:05 2020][8592490.326754] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:05 2020][8592490.332988] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.339697] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:05 2020][8592490.345656] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.352303] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.359354] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:05 2020][8592490.364881] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.371974] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.379728] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:05 2020][8592490.386986] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.394846] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.401827] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:05 2020][8592490.407257] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:05 2020][8592490.413731] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.421304] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592490.426364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.432641] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592490.439261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.445537] ll_ost_io00_030 D ffff9c5991749040 0 68783 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592490.452821] Call Trace: [Tue Mar 17 18:00:06 2020][8592490.455449] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592490.460596] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.467666] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.474803] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:06 2020][8592490.480821] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:06 2020][8592490.487009] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:06 2020][8592490.493805] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:06 2020][8592490.500022] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.506762] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.513410] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.520461] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:06 2020][8592490.526515] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.533944] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.541029] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.548778] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:06 2020][8592490.556032] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.563890] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.570857] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:06 2020][8592490.576295] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.582768] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.590338] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592490.595401] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.601675] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592490.608296] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.614571] ll_ost02_071 S ffff9c599174d140 0 68784 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592490.621848] Call Trace: [Tue Mar 17 18:00:06 2020][8592490.624478] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:06 2020][8592490.630580] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592490.635756] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.642635] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:06 2020][8592490.648418] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.654907] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.662479] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592490.667534] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.673810] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592490.680431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.686703] ll_ost00_064 S ffff9c5a36f78000 0 68785 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592490.693972] Call Trace: [Tue Mar 17 18:00:06 2020][8592490.696602] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:06 2020][8592490.702705] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592490.707880] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.714774] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:06 2020][8592490.720549] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.727025] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.734598] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592490.739657] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.745926] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592490.752545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.758821] ll_ost_io01_028 D ffff9c5a36f79040 0 68786 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592490.766098] Call Trace: [Tue Mar 17 18:00:06 2020][8592490.768726] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592490.773878] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:06 2020][8592490.780949] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:06 2020][8592490.786964] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592490.794191] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:06 2020][8592490.800033] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:06 2020][8592490.806748] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.814062] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:06 2020][8592490.820252] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592490.827050] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.834368] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.841966] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592490.849105] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:06 2020][8592490.855900] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:06 2020][8592490.862119] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.868827] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:06 2020][8592490.874868] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.882301] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.889381] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.897127] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:06 2020][8592490.904382] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.912257] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.919223] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:06 2020][8592490.924654] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.931126] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.938698] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592490.943760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.950033] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592490.956647] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592490.962922] ll_ost02_072 S ffff9c5a36f7a080 0 68787 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592490.970199] Call Trace: [Tue Mar 17 18:00:06 2020][8592490.972828] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:06 2020][8592490.978929] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592490.984122] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592490.991001] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:06 2020][8592490.996778] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.003250] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.010824] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592491.015887] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592491.022160] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592491.028780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592491.035068] ll_ost_io02_031 D ffff9c5a36f7b0c0 0 68788 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592491.042340] Call Trace: [Tue Mar 17 18:00:06 2020][8592491.044980] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.052642] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592491.057790] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.064842] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:06 2020][8592491.070860] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.078089] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.084799] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.092543] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:06 2020][8592491.098749] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.105548] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.112863] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.120448] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.127593] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:06 2020][8592491.133870] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:06 2020][8592491.140664] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:06 2020][8592491.146882] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.153618] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.160268] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.167334] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:06 2020][8592491.172853] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.179933] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.187687] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:06 2020][8592491.194943] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.202801] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.209767] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:06 2020][8592491.215195] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.221670] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.229243] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592491.234318] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592491.240586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:06 2020][8592491.247207] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:06 2020][8592491.253483] ll_ost_io03_028 D ffff9c8a334c5140 0 68789 2 0x00000080 [Tue Mar 17 18:00:06 2020][8592491.260759] Call Trace: [Tue Mar 17 18:00:06 2020][8592491.263390] [] schedule+0x29/0x70 [Tue Mar 17 18:00:06 2020][8592491.268555] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.275608] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:06 2020][8592491.281617] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.288846] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.295583] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.303325] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:06 2020][8592491.309515] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:06 2020][8592491.316313] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.323630] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.331211] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:06 2020][8592491.338351] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:06 2020][8592491.344625] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:06 2020][8592491.351159] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:06 2020][8592491.357470] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.364581] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.372334] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:06 2020][8592491.379589] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.387446] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.394413] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:06 2020][8592491.399850] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.406323] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:06 2020][8592491.413895] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:06 2020][8592491.418955] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.425221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.431858] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.438126] ll_ost_io02_032 D ffff9c8a334c1040 0 68790 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.445404] Call Trace: [Tue Mar 17 18:00:07 2020][8592491.448032] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592491.454648] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:07 2020][8592491.461009] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:07 2020][8592491.466330] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:07 2020][8592491.471831] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.478808] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.486211] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:07 2020][8592491.492246] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:07 2020][8592491.498607] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:07 2020][8592491.505225] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:07 2020][8592491.510721] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:07 2020][8592491.515955] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:07 2020][8592491.523095] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:07 2020][8592491.528331] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:07 2020][8592491.534005] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.541412] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:07 2020][8592491.548384] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.555721] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.562603] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592491.569919] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:07 2020][8592491.576798] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:07 2020][8592491.583015] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.589757] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.596404] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.603475] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:07 2020][8592491.610786] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:07 2020][8592491.616542] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:07 2020][8592491.622082] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.629164] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.636912] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:07 2020][8592491.644164] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.652025] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.658991] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:07 2020][8592491.664426] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.670901] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.678474] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592491.683535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.689825] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.696447] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.702714] ll_ost01_072 S ffff9c597fc00000 0 68791 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.709990] Call Trace: [Tue Mar 17 18:00:07 2020][8592491.712618] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592491.718712] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592491.723890] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.730769] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592491.736550] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.743025] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.750615] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592491.755677] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.761951] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.768572] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.774846] ll_ost02_073 S ffff9c597fc01040 0 68792 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.782113] Call Trace: [Tue Mar 17 18:00:07 2020][8592491.784744] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592491.790847] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592491.796024] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.802901] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592491.808698] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.815166] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.822738] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592491.827792] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.834068] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.840689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.846964] ll_ost02_074 S ffff9c597fc02080 0 68793 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.854248] Call Trace: [Tue Mar 17 18:00:07 2020][8592491.856877] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592491.862981] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592491.868170] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.875051] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592491.880828] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.887300] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.894871] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592491.899925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.906200] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.912822] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.919097] ll_ost00_065 S ffff9c597fc030c0 0 68794 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.926372] Call Trace: [Tue Mar 17 18:00:07 2020][8592491.929002] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592491.935105] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592491.940297] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.947175] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592491.952952] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.959422] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592491.966996] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592491.972052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.978325] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592491.984950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592491.991234] ll_ost00_066 S ffff9c597fc04100 0 68795 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592491.998505] Call Trace: [Tue Mar 17 18:00:07 2020][8592492.001134] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592492.007237] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592492.012416] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.019293] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592492.025075] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.031549] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.039120] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592492.044175] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.050463] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592492.057078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.063353] ll_ost01_073 S ffff9c597fc05140 0 68796 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592492.070621] Call Trace: [Tue Mar 17 18:00:07 2020][8592492.073250] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592492.079344] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592492.084521] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.091400] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592492.097183] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.103657] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.111244] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592492.116298] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.122572] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592492.129195] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.135470] ll_ost02_075 D ffff9c597fc06180 0 68797 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592492.142745] Call Trace: [Tue Mar 17 18:00:07 2020][8592492.145375] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592492.150527] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:07 2020][8592492.157578] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:07 2020][8592492.163597] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:07 2020][8592492.170836] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:07 2020][8592492.177547] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592492.185034] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:07 2020][8592492.191225] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:07 2020][8592492.198024] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592492.205340] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:07 2020][8592492.212920] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:07 2020][8592492.220063] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:07 2020][8592492.226335] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:07 2020][8592492.232436] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:07 2020][8592492.239182] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.246949] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.254959] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:07 2020][8592492.261439] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.268527] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.276289] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:07 2020][8592492.283543] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.291404] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.298399] [] ? ptlrpc_server_post_idle_rqbds+0x7e/0xf0 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.306346] [] ? ptlrpc_grow_req_bufs+0x50/0x2a0 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.313614] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.320089] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.327663] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592492.332724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.339000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:07 2020][8592492.345622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.351894] ll_ost02_076 S ffff9c597f118000 0 68798 2 0x00000080 [Tue Mar 17 18:00:07 2020][8592492.359184] Call Trace: [Tue Mar 17 18:00:07 2020][8592492.361817] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:07 2020][8592492.367910] [] schedule+0x29/0x70 [Tue Mar 17 18:00:07 2020][8592492.373102] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.379985] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:07 2020][8592492.386605] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:07 2020][8592492.392389] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.398864] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:07 2020][8592492.406436] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:07 2020][8592492.411497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:07 2020][8592492.417772] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592492.424392] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.430667] ll_ost03_066 S ffff9c597f119040 0 68799 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592492.437961] Call Trace: [Tue Mar 17 18:00:08 2020][8592492.440589] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:08 2020][8592492.446683] [] schedule+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592492.451862] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.458738] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:08 2020][8592492.464521] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.470995] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.478568] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592492.483629] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.489904] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592492.496541] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.502809] ll_ost01_074 S ffff9c597f11a080 0 68800 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592492.510094] Call Trace: [Tue Mar 17 18:00:08 2020][8592492.512724] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:08 2020][8592492.518824] [] schedule+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592492.524003] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.530883] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:08 2020][8592492.536664] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.543137] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.550709] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592492.555777] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.562044] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592492.568660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.574932] ll_ost_io00_031 D ffff9c597f11b0c0 0 68801 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592492.582200] Call Trace: [Tue Mar 17 18:00:08 2020][8592492.584831] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592492.591451] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:08 2020][8592492.597813] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:08 2020][8592492.603135] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:08 2020][8592492.608638] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.615627] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.623025] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:08 2020][8592492.629046] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:08 2020][8592492.635405] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:08 2020][8592492.642023] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:08 2020][8592492.649163] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:08 2020][8592492.654397] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:08 2020][8592492.660074] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.667481] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:08 2020][8592492.674452] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.681787] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.688671] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.695986] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:08 2020][8592492.702867] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:08 2020][8592492.709084] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.715823] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.722470] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.729542] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:08 2020][8592492.736857] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:08 2020][8592492.742642] [] ? tgt_brw_read+0x1e50/0x1e50 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.749484] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.756566] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.764315] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:08 2020][8592492.771567] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.779428] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.786393] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:08 2020][8592492.791830] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.798302] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.805876] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592492.810937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.817228] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592492.823851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592492.830126] ll_ost_io02_033 D ffff9c597f11c100 0 68802 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592492.837401] Call Trace: [Tue Mar 17 18:00:08 2020][8592492.840031] [] schedule+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592492.845182] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:08 2020][8592492.852233] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:08 2020][8592492.858252] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:08 2020][8592492.865478] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:08 2020][8592492.871323] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:08 2020][8592492.878047] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.885358] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:08 2020][8592492.891550] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:08 2020][8592492.898346] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.905663] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.913245] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592492.920385] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:08 2020][8592492.927179] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:08 2020][8592492.933397] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.940107] [] ? load_balance+0x1be/0x9a0 [Tue Mar 17 18:00:08 2020][8592492.945967] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:08 2020][8592492.951718] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:08 2020][8592492.958272] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.965706] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.972794] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.980539] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:08 2020][8592492.987794] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592492.995655] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.002619] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:08 2020][8592493.008057] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.014546] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.022118] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592493.027181] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.033457] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592493.040076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.046352] ll_ost_io02_034 D ffff9c597f11d140 0 68803 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592493.053628] Call Trace: [Tue Mar 17 18:00:08 2020][8592493.056256] [] ? find_busiest_group+0x144/0x990 [Tue Mar 17 18:00:08 2020][8592493.062619] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592493.069240] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:08 2020][8592493.075616] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:08 2020][8592493.080933] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:08 2020][8592493.086432] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.093410] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.100812] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:08 2020][8592493.106833] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:08 2020][8592493.113195] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:08 2020][8592493.119815] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:08 2020][8592493.126952] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:08 2020][8592493.132187] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:08 2020][8592493.137874] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.145274] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:08 2020][8592493.152247] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.159561] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.166444] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:08 2020][8592493.173766] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:08 2020][8592493.180650] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:08 2020][8592493.186873] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.193583] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:08 2020][8592493.199623] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.207069] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.214152] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.221901] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:08 2020][8592493.229153] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.237012] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.243978] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:08 2020][8592493.249414] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.255890] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.263461] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592493.268524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.274827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592493.281444] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.287718] ll_ost02_077 D ffff9c597f11e180 0 68804 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592493.294986] Call Trace: [Tue Mar 17 18:00:08 2020][8592493.297616] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:08 2020][8592493.303631] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:08 2020][8592493.310255] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:08 2020][8592493.316614] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:08 2020][8592493.321973] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.328971] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.336056] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.343805] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:08 2020][8592493.351058] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.358920] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.365906] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:08 2020][8592493.371337] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.377811] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:08 2020][8592493.385384] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:08 2020][8592493.390447] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.396720] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:08 2020][8592493.403357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:08 2020][8592493.409626] ll_ost00_067 S ffff9c597da88000 0 68805 2 0x00000080 [Tue Mar 17 18:00:08 2020][8592493.416901] Call Trace: [Tue Mar 17 18:00:08 2020][8592493.419531] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:09 2020][8592493.425633] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592493.430811] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.437689] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:09 2020][8592493.443473] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.449948] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.457524] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592493.462593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.468862] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592493.475484] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.481760] ll_ost00_068 S ffff9c597da89040 0 68806 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592493.489044] Call Trace: [Tue Mar 17 18:00:09 2020][8592493.491674] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:09 2020][8592493.497775] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592493.502955] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.509837] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:09 2020][8592493.515614] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.522087] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.529675] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592493.534729] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.541004] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592493.547627] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.553902] ll_ost00_069 S ffff9c597da8a080 0 68807 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592493.561179] Call Trace: [Tue Mar 17 18:00:09 2020][8592493.563806] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:09 2020][8592493.569908] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592493.575086] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.581978] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:09 2020][8592493.587754] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.594231] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.601803] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592493.606864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.613138] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592493.619760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.626033] ll_ost00_070 D ffff9c597da8b0c0 0 68808 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592493.633301] Call Trace: [Tue Mar 17 18:00:09 2020][8592493.635933] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:09 2020][8592493.642554] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592493.647719] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.654772] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:09 2020][8592493.660781] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.668014] [] ? kiblnd_send+0x357/0xa20 [ko2iblnd] [Tue Mar 17 18:00:09 2020][8592493.674725] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.681456] [] ? lu_obj_hop_keycmp+0x1d/0x30 [obdclass] [Tue Mar 17 18:00:09 2020][8592493.688519] [] ? cfs_hash_bd_lookup_intent+0x63/0x170 [libcfs] [Tue Mar 17 18:00:09 2020][8592493.696176] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:09 2020][8592493.702363] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.709178] [] ? ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.716495] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.724079] [] ldiskfs_acquire_dquot+0x53/0xb0 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.731222] [] dqget+0x3fa/0x450 [Tue Mar 17 18:00:09 2020][8592493.736284] [] dquot_get_dqblk+0x14/0x1f0 [Tue Mar 17 18:00:09 2020][8592493.742134] [] osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.749798] [] lquotactl_slv+0x27d/0x9d0 [lquota] [Tue Mar 17 18:00:09 2020][8592493.756332] [] ofd_quotactl+0x13c/0x380 [ofd] [Tue Mar 17 18:00:09 2020][8592493.762551] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.769639] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.777400] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:09 2020][8592493.784655] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.792517] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.799480] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:09 2020][8592493.805256] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.811730] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.819302] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592493.824364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.830640] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592493.837260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592493.843549] ll_ost01_075 D ffff9c597da8c100 0 68809 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592493.850818] Call Trace: [Tue Mar 17 18:00:09 2020][8592493.853449] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592493.858600] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.865652] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:09 2020][8592493.871668] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.878898] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.885609] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.893101] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:09 2020][8592493.899291] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592493.906104] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.913420] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.920996] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592493.928143] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:09 2020][8592493.934418] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:09 2020][8592493.940519] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:09 2020][8592493.947250] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.955023] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.963034] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:09 2020][8592493.969512] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.976616] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.984365] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:09 2020][8592493.991618] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592493.999477] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.006444] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:09 2020][8592494.011879] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.018352] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.025925] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592494.030987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592494.037262] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592494.043897] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592494.050166] ll_ost01_076 D ffff9c597da8d140 0 68810 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592494.057442] Call Trace: [Tue Mar 17 18:00:09 2020][8592494.060071] [] schedule+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592494.065214] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:09 2020][8592494.072266] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:09 2020][8592494.078284] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592494.085513] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:09 2020][8592494.092224] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.099715] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:09 2020][8592494.105918] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592494.112720] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.120036] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.127618] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.134766] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:09 2020][8592494.141041] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:09 2020][8592494.147143] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:09 2020][8592494.153872] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.161648] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.169659] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:09 2020][8592494.176150] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.183231] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.190980] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:09 2020][8592494.198233] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.206094] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.213059] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:09 2020][8592494.218495] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.224971] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.232547] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:09 2020][8592494.237601] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592494.243876] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:09 2020][8592494.250512] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:09 2020][8592494.256779] ll_ost_io03_029 D ffff9c597da8e180 0 68811 2 0x00000080 [Tue Mar 17 18:00:09 2020][8592494.264048] Call Trace: [Tue Mar 17 18:00:09 2020][8592494.266678] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:09 2020][8592494.273314] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:09 2020][8592494.279669] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:09 2020][8592494.285004] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:09 2020][8592494.290501] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.297477] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.304889] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:09 2020][8592494.310901] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:09 2020][8592494.317262] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:09 2020][8592494.323885] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.330596] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:09 2020][8592494.337736] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:09 2020][8592494.342962] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:09 2020][8592494.348638] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.356045] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:09 2020][8592494.363034] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.370369] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.377255] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:09 2020][8592494.384576] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:09 2020][8592494.391458] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:09 2020][8592494.397674] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.404414] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.411063] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:09 2020][8592494.418130] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:10 2020][8592494.425446] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:10 2020][8592494.431199] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:10 2020][8592494.437774] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.445203] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.452289] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.460037] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:10 2020][8592494.467289] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.475151] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.482117] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:10 2020][8592494.487554] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.494028] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.501599] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592494.506675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.512946] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592494.519562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.525831] ll_ost00_071 S ffff9c5918538000 0 68812 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592494.533117] Call Trace: [Tue Mar 17 18:00:10 2020][8592494.535744] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:10 2020][8592494.541840] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592494.547017] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.553894] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:10 2020][8592494.560516] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:10 2020][8592494.566313] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.572790] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.580361] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592494.585425] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.591699] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592494.598320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.604594] ll_ost00_072 S ffff9c5918539040 0 68813 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592494.611869] Call Trace: [Tue Mar 17 18:00:10 2020][8592494.614501] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:10 2020][8592494.620602] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592494.625794] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.632673] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:10 2020][8592494.638448] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.644922] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.652496] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592494.657557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.663832] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592494.670453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.676728] ll_ost00_073 S ffff9c591853a080 0 68814 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592494.684002] Call Trace: [Tue Mar 17 18:00:10 2020][8592494.686632] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:10 2020][8592494.692725] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592494.697917] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.704798] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:10 2020][8592494.711420] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:10 2020][8592494.717203] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.723678] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.731249] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592494.736311] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.742586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592494.749221] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.755492] ll_ost01_077 S ffff9c591853b0c0 0 68815 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592494.762766] Call Trace: [Tue Mar 17 18:00:10 2020][8592494.765395] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:10 2020][8592494.771490] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592494.776667] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.783545] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:10 2020][8592494.789327] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.795803] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.803373] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592494.808450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.814718] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592494.821340] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592494.827615] ll_ost_io03_030 D ffff9c591853c100 0 68816 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592494.834892] Call Trace: [Tue Mar 17 18:00:10 2020][8592494.837522] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592494.844143] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:10 2020][8592494.850504] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:10 2020][8592494.855827] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:10 2020][8592494.861327] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.868317] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.875715] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:10 2020][8592494.881734] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:10 2020][8592494.888098] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:10 2020][8592494.894723] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:10 2020][8592494.900215] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:10 2020][8592494.905452] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:10 2020][8592494.912592] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:10 2020][8592494.917827] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:10 2020][8592494.923499] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.930913] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:10 2020][8592494.937888] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.945202] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.952085] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592494.959406] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:10 2020][8592494.966287] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:10 2020][8592494.972506] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.979242] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.985892] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592494.992942] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:10 2020][8592494.998483] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.005566] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.013314] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:10 2020][8592495.020567] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.028425] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.035391] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:10 2020][8592495.040828] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.047301] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.054874] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592495.059936] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592495.066225] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592495.072840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592495.079115] ll_ost_io01_029 D ffff9c591853d140 0 68817 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592495.086391] Call Trace: [Tue Mar 17 18:00:10 2020][8592495.089021] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592495.094173] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.101224] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:10 2020][8592495.107241] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.114466] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:10 2020][8592495.120310] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.127039] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.134349] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:10 2020][8592495.140538] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.147336] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.154655] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.162235] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.169383] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:10 2020][8592495.176178] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:10 2020][8592495.182395] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.189106] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:10 2020][8592495.195157] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.202586] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.209666] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.217415] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:10 2020][8592495.224667] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.232529] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.239494] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:10 2020][8592495.244929] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.251404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.258976] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:10 2020][8592495.264050] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592495.270321] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:10 2020][8592495.276963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:10 2020][8592495.283233] ll_ost_io00_032 D ffff9c591853e180 0 68818 2 0x00000080 [Tue Mar 17 18:00:10 2020][8592495.290509] Call Trace: [Tue Mar 17 18:00:10 2020][8592495.293140] [] schedule+0x29/0x70 [Tue Mar 17 18:00:10 2020][8592495.298289] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.305341] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:10 2020][8592495.311362] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.318587] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.325314] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.333057] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:10 2020][8592495.339242] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:10 2020][8592495.346038] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.353354] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.360937] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:10 2020][8592495.368078] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:10 2020][8592495.374350] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:10 2020][8592495.381145] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:10 2020][8592495.387363] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.394118] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.400768] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:10 2020][8592495.407837] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:10 2020][8592495.415151] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:10 2020][8592495.420904] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:11 2020][8592495.427466] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.434898] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.441980] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.449732] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:11 2020][8592495.456989] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.464847] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.471830] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:11 2020][8592495.477268] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.483741] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.491313] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592495.496373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.502650] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592495.509272] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.515546] ll_ost00_074 S ffff9c5917d88000 0 68819 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592495.522831] Call Trace: [Tue Mar 17 18:00:11 2020][8592495.525460] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:11 2020][8592495.531562] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592495.536753] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.543636] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:11 2020][8592495.549419] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.555892] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.563464] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592495.568524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.574800] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592495.581422] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.587709] ll_ost00_075 S ffff9c5917d89040 0 68820 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592495.594980] Call Trace: [Tue Mar 17 18:00:11 2020][8592495.597609] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:11 2020][8592495.603704] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592495.608879] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.615758] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:11 2020][8592495.621543] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.628016] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.635587] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592495.640649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.646940] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592495.653554] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.659829] ll_ost00_076 S ffff9c5917d8a080 0 68821 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592495.667104] Call Trace: [Tue Mar 17 18:00:11 2020][8592495.669734] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:11 2020][8592495.675837] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592495.681014] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.687893] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:11 2020][8592495.693676] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.700150] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.707739] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592495.712801] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.719074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592495.725695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592495.731969] ll_ost_io01_030 D ffff9c597f6e0000 0 68822 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592495.739239] Call Trace: [Tue Mar 17 18:00:11 2020][8592495.741869] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592495.748491] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:11 2020][8592495.754850] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:11 2020][8592495.760170] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:11 2020][8592495.765679] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.772650] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.780055] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:11 2020][8592495.786337] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.793225] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.801577] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:00:11 2020][8592495.807944] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.815178] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.822843] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:11 2020][8592495.828431] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.836113] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.844038] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.851447] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.858506] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.866691] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:11 2020][8592495.873662] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.880897] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:11 2020][8592495.888210] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:11 2020][8592495.895003] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:11 2020][8592495.901221] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.907978] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.914634] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.921702] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:11 2020][8592495.929019] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:11 2020][8592495.934772] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:11 2020][8592495.941333] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.948768] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.955855] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.963602] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:11 2020][8592495.970854] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.978731] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.985698] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:11 2020][8592495.991135] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592495.997609] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.005181] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592496.010241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.016516] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592496.023138] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.029412] ll_ost00_077 S ffff9c597f6e1040 0 68823 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592496.036680] Call Trace: [Tue Mar 17 18:00:11 2020][8592496.039309] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:11 2020][8592496.045412] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592496.050605] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.057484] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:11 2020][8592496.063269] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.069743] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.077315] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592496.082376] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.088650] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592496.095270] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.101551] ll_ost01_078 D ffff9c597f6e2080 0 68824 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592496.108821] Call Trace: [Tue Mar 17 18:00:11 2020][8592496.111452] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:11 2020][8592496.116773] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592496.121921] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:11 2020][8592496.127941] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:11 2020][8592496.135165] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:11 2020][8592496.140918] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:11 2020][8592496.147456] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:11 2020][8592496.153296] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:11 2020][8592496.158632] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:11 2020][8592496.164813] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:11 2020][8592496.170220] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:11 2020][8592496.175628] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:00:11 2020][8592496.181211] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:11 2020][8592496.186532] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:00:11 2020][8592496.192980] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.199085] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:00:11 2020][8592496.205965] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:00:11 2020][8592496.211373] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:11 2020][8592496.218971] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.227244] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.234905] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:11 2020][8592496.241876] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.249024] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.256514] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.263492] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 18:00:11 2020][8592496.270291] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 18:00:11 2020][8592496.276407] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:11 2020][8592496.283141] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.290913] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.298940] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:11 2020][8592496.305420] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.312508] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.320256] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:11 2020][8592496.327510] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.335383] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.342350] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:11 2020][8592496.347788] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.354261] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.361847] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:11 2020][8592496.366903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.373178] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:11 2020][8592496.379799] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:11 2020][8592496.386075] ll_ost00_078 S ffff9c597f6e30c0 0 68825 2 0x00000080 [Tue Mar 17 18:00:11 2020][8592496.393351] Call Trace: [Tue Mar 17 18:00:11 2020][8592496.395980] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:11 2020][8592496.402074] [] schedule+0x29/0x70 [Tue Mar 17 18:00:11 2020][8592496.407251] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.414128] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:11 2020][8592496.419928] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:11 2020][8592496.426404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.433975] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592496.439039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.445311] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592496.451934] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.458210] ll_ost01_079 S ffff9c597f6e4100 0 68826 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592496.465484] Call Trace: [Tue Mar 17 18:00:12 2020][8592496.468114] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:12 2020][8592496.474215] [] schedule+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592496.479408] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.486287] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:12 2020][8592496.492062] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.498538] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.506110] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592496.511171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.517446] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592496.524065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.530340] ll_ost01_080 S ffff9c597f6e5140 0 68827 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592496.537608] Call Trace: [Tue Mar 17 18:00:12 2020][8592496.540239] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:12 2020][8592496.546340] [] schedule+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592496.551532] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.558414] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:12 2020][8592496.564196] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.570670] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.578243] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592496.583304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.589579] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592496.596200] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.602488] ll_ost02_078 D ffff9c597f6e6180 0 68828 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592496.609758] Call Trace: [Tue Mar 17 18:00:12 2020][8592496.612388] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592496.619009] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:12 2020][8592496.625373] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:12 2020][8592496.630692] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:12 2020][8592496.636196] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.643170] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.650566] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 18:00:12 2020][8592496.656493] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 18:00:12 2020][8592496.662784] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:12 2020][8592496.668104] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.675071] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.682560] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.689875] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.697537] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.704675] [] evict+0xb4/0x180 [Tue Mar 17 18:00:12 2020][8592496.709647] [] iput+0xfc/0x190 [Tue Mar 17 18:00:12 2020][8592496.714541] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.721871] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:00:12 2020][8592496.729289] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:00:12 2020][8592496.736446] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:00:12 2020][8592496.743069] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:00:12 2020][8592496.749801] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.757574] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.765585] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:12 2020][8592496.772065] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.779150] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.786897] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:12 2020][8592496.794151] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.802026] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.808993] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:12 2020][8592496.815617] [] ? _raw_spin_lock+0x20/0x30 [Tue Mar 17 18:00:12 2020][8592496.821487] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.827963] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592496.835537] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592496.840594] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.846867] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592496.853490] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592496.859763] ll_ost_io03_031 D ffff9c59f8b78000 0 68829 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592496.867056] Call Trace: [Tue Mar 17 18:00:12 2020][8592496.869687] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592496.876308] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:12 2020][8592496.882669] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:12 2020][8592496.887991] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:12 2020][8592496.893491] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.900461] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.907865] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:12 2020][8592496.914145] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.921037] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.929413] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.936644] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.944308] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:12 2020][8592496.949896] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.957565] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.965496] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.972906] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.979971] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592496.988156] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:12 2020][8592496.995128] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.002363] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.009692] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:12 2020][8592497.016488] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:12 2020][8592497.022705] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.029445] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.036093] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.043158] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:12 2020][8592497.050475] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:12 2020][8592497.056231] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:12 2020][8592497.061756] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.068862] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.076609] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:12 2020][8592497.083864] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.091722] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.098689] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:12 2020][8592497.104126] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.110599] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.118171] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592497.123231] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.129508] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592497.136144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.142410] ll_ost_io00_033 D ffff9c59f8b79040 0 68830 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592497.149677] Call Trace: [Tue Mar 17 18:00:12 2020][8592497.152307] [] schedule+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592497.157452] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:12 2020][8592497.164504] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:12 2020][8592497.170519] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:12 2020][8592497.177750] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:12 2020][8592497.184460] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.192210] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:12 2020][8592497.198415] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:12 2020][8592497.205215] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.212532] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.220114] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:12 2020][8592497.227254] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:12 2020][8592497.233528] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:12 2020][8592497.240063] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:12 2020][8592497.246369] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.253455] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.261201] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:12 2020][8592497.268469] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.276331] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.283313] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:12 2020][8592497.288743] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.295236] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.302806] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592497.307869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.314143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592497.320765] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.327039] ll_ost01_081 S ffff9c59f8b7a080 0 68831 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592497.334333] Call Trace: [Tue Mar 17 18:00:12 2020][8592497.336965] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:12 2020][8592497.343062] [] schedule+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592497.348242] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.355148] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:12 2020][8592497.360928] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.367402] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:12 2020][8592497.374977] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:12 2020][8592497.380036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.386311] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:12 2020][8592497.392946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:12 2020][8592497.399215] ll_ost_io02_035 D ffff9c59f8b7b0c0 0 68832 2 0x00000080 [Tue Mar 17 18:00:12 2020][8592497.406492] Call Trace: [Tue Mar 17 18:00:12 2020][8592497.409120] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:12 2020][8592497.415743] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:12 2020][8592497.422105] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:12 2020][8592497.427425] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:13 2020][8592497.432927] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.439904] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.447309] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:13 2020][8592497.453341] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:13 2020][8592497.459704] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:13 2020][8592497.466323] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:13 2020][8592497.471815] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:13 2020][8592497.477042] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:13 2020][8592497.484185] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:13 2020][8592497.489418] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:13 2020][8592497.495091] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.502492] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592497.509462] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.516791] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.523674] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.530989] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:13 2020][8592497.537871] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:13 2020][8592497.544089] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.550827] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.557474] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.564525] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:13 2020][8592497.570053] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.577139] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.584901] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:13 2020][8592497.592156] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.600018] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.606984] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:13 2020][8592497.612419] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.618892] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.626465] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:13 2020][8592497.631526] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592497.637802] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:13 2020][8592497.644423] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592497.650710] ll_ost_io00_034 D ffff9c59f8b7c100 0 68833 2 0x00000080 [Tue Mar 17 18:00:13 2020][8592497.657983] Call Trace: [Tue Mar 17 18:00:13 2020][8592497.660612] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:13 2020][8592497.665933] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:13 2020][8592497.672556] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:13 2020][8592497.678917] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:13 2020][8592497.684237] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:13 2020][8592497.689739] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.696715] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.704119] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:13 2020][8592497.710416] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.718768] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:00:13 2020][8592497.725135] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.732371] [] ? ldiskfs_es_insert_extent+0xdb/0x1c0 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.740034] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:13 2020][8592497.745623] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.753292] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.760952] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.768883] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.776283] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.783341] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.791541] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592497.798514] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.805743] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592497.813065] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:13 2020][8592497.819860] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:13 2020][8592497.826079] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.832815] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.839464] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.846514] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:13 2020][8592497.852569] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.860000] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.867085] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.874833] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:13 2020][8592497.882086] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.889946] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.896911] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:13 2020][8592497.902350] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.908822] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.916395] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:13 2020][8592497.921470] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592497.927737] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:13 2020][8592497.934358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592497.940635] ldlm_cn03_004 S ffff9c59f8b7d140 0 68834 2 0x00000080 [Tue Mar 17 18:00:13 2020][8592497.947911] Call Trace: [Tue Mar 17 18:00:13 2020][8592497.950538] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:13 2020][8592497.956633] [] schedule+0x29/0x70 [Tue Mar 17 18:00:13 2020][8592497.961810] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.968687] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:13 2020][8592497.974472] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.980962] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592497.988534] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:13 2020][8592497.993588] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592497.999863] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:13 2020][8592498.006485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592498.012758] ll_ost_io02_036 D ffff9c59f8b7e180 0 68835 2 0x00000080 [Tue Mar 17 18:00:13 2020][8592498.020025] Call Trace: [Tue Mar 17 18:00:13 2020][8592498.022656] [] schedule+0x29/0x70 [Tue Mar 17 18:00:13 2020][8592498.027809] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.034860] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:13 2020][8592498.040877] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.048118] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.054832] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.062576] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:13 2020][8592498.068764] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.075563] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.082880] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.090462] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.097600] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:13 2020][8592498.103875] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:13 2020][8592498.110686] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:13 2020][8592498.116905] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.123617] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:13 2020][8592498.129655] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.137089] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.144169] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.151915] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:13 2020][8592498.159169] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.167030] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.173995] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:13 2020][8592498.179449] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.185922] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.193495] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:13 2020][8592498.198556] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592498.204830] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:13 2020][8592498.211453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592498.217728] ll_ost_io02_037 D ffff9c597d9f0000 0 68836 2 0x00000080 [Tue Mar 17 18:00:13 2020][8592498.225004] Call Trace: [Tue Mar 17 18:00:13 2020][8592498.227641] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.235303] [] schedule+0x29/0x70 [Tue Mar 17 18:00:13 2020][8592498.240470] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.247522] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:13 2020][8592498.253531] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.260758] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.267471] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.275214] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:13 2020][8592498.281419] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:13 2020][8592498.288218] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.295549] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.303127] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:13 2020][8592498.310274] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:13 2020][8592498.316564] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:13 2020][8592498.323359] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:13 2020][8592498.329577] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.336307] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.342958] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.350014] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:13 2020][8592498.355552] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.362641] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.370397] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:13 2020][8592498.377664] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.385527] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.392491] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:13 2020][8592498.397930] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.404404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:13 2020][8592498.411975] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:13 2020][8592498.417037] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:13 2020][8592498.423311] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:13 2020][8592498.429932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592498.436206] ll_ost_io02_038 D ffff9c597d9f1040 0 68837 2 0x00000080 [Tue Mar 17 18:00:14 2020][8592498.443507] Call Trace: [Tue Mar 17 18:00:14 2020][8592498.446139] [] schedule+0x29/0x70 [Tue Mar 17 18:00:14 2020][8592498.451291] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.458340] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:14 2020][8592498.464359] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.471585] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.478298] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.486040] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:14 2020][8592498.492231] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.499030] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.506367] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.513943] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.521085] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:14 2020][8592498.527358] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:14 2020][8592498.534154] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:14 2020][8592498.540371] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.547109] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.553757] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.560827] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:14 2020][8592498.568140] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:14 2020][8592498.573910] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:14 2020][8592498.580464] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.587897] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.594979] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.602731] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:14 2020][8592498.609986] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.617844] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.624811] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:14 2020][8592498.630249] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.636723] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.644312] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:14 2020][8592498.649372] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592498.655648] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:14 2020][8592498.662270] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592498.668544] ll_ost_io00_035 D ffff9c597d9f2080 0 68838 2 0x00000080 [Tue Mar 17 18:00:14 2020][8592498.675823] Call Trace: [Tue Mar 17 18:00:14 2020][8592498.678449] [] schedule+0x29/0x70 [Tue Mar 17 18:00:14 2020][8592498.683592] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.690646] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:14 2020][8592498.696661] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.703902] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:14 2020][8592498.709740] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.716454] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.723770] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:14 2020][8592498.729962] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.736755] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.744074] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.751655] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.758803] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:14 2020][8592498.765599] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:14 2020][8592498.771832] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.778563] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.785211] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.792297] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:14 2020][8592498.799611] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:14 2020][8592498.805139] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.812227] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.819975] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:14 2020][8592498.827227] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.835088] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.842094] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.848563] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.856135] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:14 2020][8592498.861198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592498.867472] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:14 2020][8592498.874093] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592498.880367] ll_ost_io01_031 D ffff9c597d9f30c0 0 68839 2 0x00000080 [Tue Mar 17 18:00:14 2020][8592498.887643] Call Trace: [Tue Mar 17 18:00:14 2020][8592498.890273] [] schedule+0x29/0x70 [Tue Mar 17 18:00:14 2020][8592498.895425] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.902490] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:14 2020][8592498.908502] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.915730] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.922441] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.930183] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:14 2020][8592498.936374] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592498.943173] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.950498] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.958079] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592498.965219] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:14 2020][8592498.971508] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:14 2020][8592498.978304] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:14 2020][8592498.984522] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.991261] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592498.997909] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.004982] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:14 2020][8592499.012293] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:14 2020][8592499.018046] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:14 2020][8592499.024608] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.032041] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.039137] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.046887] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:14 2020][8592499.054140] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.062000] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.068966] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:14 2020][8592499.074400] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.080875] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.088445] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:14 2020][8592499.093499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592499.099773] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:14 2020][8592499.106411] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592499.112677] ll_ost_io01_032 D ffff9c597d9f4100 0 68840 2 0x00000080 [Tue Mar 17 18:00:14 2020][8592499.119945] Call Trace: [Tue Mar 17 18:00:14 2020][8592499.122584] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.130247] [] schedule+0x29/0x70 [Tue Mar 17 18:00:14 2020][8592499.135400] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.142448] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:14 2020][8592499.148468] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.155695] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.162406] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.170163] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:14 2020][8592499.176347] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.183144] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.190460] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.198037] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.205183] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:14 2020][8592499.211457] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:14 2020][8592499.218254] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:14 2020][8592499.224479] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.231187] [] ? ___slab_alloc+0x24e/0x4f0 [Tue Mar 17 18:00:14 2020][8592499.237159] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.243810] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.250863] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:14 2020][8592499.256615] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:14 2020][8592499.262142] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.269224] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.276980] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:14 2020][8592499.284247] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.292109] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.299075] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:14 2020][8592499.304540] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.311012] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:14 2020][8592499.318584] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:14 2020][8592499.323644] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592499.329920] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:14 2020][8592499.336541] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:14 2020][8592499.342815] ll_ost_io02_039 D ffff9c597d9f5140 0 68841 2 0x00000080 [Tue Mar 17 18:00:14 2020][8592499.350101] Call Trace: [Tue Mar 17 18:00:14 2020][8592499.352732] [] schedule+0x29/0x70 [Tue Mar 17 18:00:14 2020][8592499.357902] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.364951] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:14 2020][8592499.370970] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.378193] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:14 2020][8592499.384036] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.390751] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.398066] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:14 2020][8592499.404256] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:14 2020][8592499.411055] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.418380] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:14 2020][8592499.425963] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.433122] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:15 2020][8592499.439921] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:15 2020][8592499.446138] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.452877] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.459527] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.466576] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:15 2020][8592499.472104] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.479192] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.486940] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:15 2020][8592499.494192] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.502068] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.509034] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:15 2020][8592499.514470] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.520946] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.528517] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:15 2020][8592499.533579] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.539854] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:15 2020][8592499.546476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.552750] ll_ost_io00_036 D ffff9c597d9f6180 0 68842 2 0x00000080 [Tue Mar 17 18:00:15 2020][8592499.560025] Call Trace: [Tue Mar 17 18:00:15 2020][8592499.562655] [] schedule+0x29/0x70 [Tue Mar 17 18:00:15 2020][8592499.567807] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.574874] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:15 2020][8592499.580893] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.588120] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:15 2020][8592499.593968] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.600682] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.607992] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:15 2020][8592499.614182] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.620980] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.628311] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.635885] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.643027] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:15 2020][8592499.649820] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:15 2020][8592499.656039] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.662748] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:15 2020][8592499.668704] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.675353] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.682405] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:15 2020][8592499.687931] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.695033] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.702785] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:15 2020][8592499.710037] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.717897] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.724863] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:15 2020][8592499.730299] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.736771] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.744344] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:15 2020][8592499.749405] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.755682] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:15 2020][8592499.762301] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.768577] ll_ost_io02_040 D ffff9c5980170000 0 68843 2 0x00000080 [Tue Mar 17 18:00:15 2020][8592499.775852] Call Trace: [Tue Mar 17 18:00:15 2020][8592499.778490] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.786154] [] schedule+0x29/0x70 [Tue Mar 17 18:00:15 2020][8592499.791306] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.798355] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:15 2020][8592499.804373] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.811601] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.818312] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.826071] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:15 2020][8592499.832261] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592499.839061] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.846387] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.853968] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592499.861115] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:15 2020][8592499.867389] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:15 2020][8592499.874185] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:15 2020][8592499.880401] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.887105] [] ? ___slab_alloc+0x24e/0x4f0 [Tue Mar 17 18:00:15 2020][8592499.893076] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.899725] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.906778] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:15 2020][8592499.912303] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.919383] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.927130] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:15 2020][8592499.934384] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.942244] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.949211] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:15 2020][8592499.954646] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.961134] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592499.968710] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:15 2020][8592499.973770] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.980047] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:15 2020][8592499.986667] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592499.992941] ll_ost_io00_037 D ffff9c5980171040 0 68844 2 0x00000080 [Tue Mar 17 18:00:15 2020][8592500.000216] Call Trace: [Tue Mar 17 18:00:15 2020][8592500.002846] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:15 2020][8592500.009468] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:15 2020][8592500.015831] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:15 2020][8592500.021167] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:15 2020][8592500.026661] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.033638] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.041041] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:15 2020][8592500.047061] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:15 2020][8592500.053422] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:15 2020][8592500.060038] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:15 2020][8592500.065532] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:15 2020][8592500.070760] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:15 2020][8592500.077900] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:15 2020][8592500.083150] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:15 2020][8592500.088818] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.096217] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592500.103187] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.110504] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.117392] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.124705] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:15 2020][8592500.131587] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:15 2020][8592500.137803] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.144507] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:15 2020][8592500.150047] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.157127] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.164876] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:15 2020][8592500.172129] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.179991] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.186964] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:15 2020][8592500.192398] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.198873] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.206446] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:15 2020][8592500.211508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592500.217797] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:15 2020][8592500.224413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:15 2020][8592500.230686] ll_ost_io01_033 D ffff9c5980172080 0 68845 2 0x00000080 [Tue Mar 17 18:00:15 2020][8592500.237955] Call Trace: [Tue Mar 17 18:00:15 2020][8592500.240587] [] schedule+0x29/0x70 [Tue Mar 17 18:00:15 2020][8592500.245735] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:15 2020][8592500.252788] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:15 2020][8592500.258806] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592500.266031] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:15 2020][8592500.271875] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:15 2020][8592500.278604] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.285928] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:15 2020][8592500.292121] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:15 2020][8592500.298917] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.306257] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.313832] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:15 2020][8592500.320977] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:15 2020][8592500.327790] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:15 2020][8592500.334009] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.340721] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:15 2020][8592500.346776] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.354202] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.361283] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.369031] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:15 2020][8592500.376282] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.384142] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.391108] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:15 2020][8592500.396545] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.403017] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:15 2020][8592500.410590] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:16 2020][8592500.415667] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.421935] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:16 2020][8592500.428558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.434831] ll_ost_io01_034 D ffff9c59801730c0 0 68846 2 0x00000080 [Tue Mar 17 18:00:16 2020][8592500.442098] Call Trace: [Tue Mar 17 18:00:16 2020][8592500.444728] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:16 2020][8592500.451350] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:16 2020][8592500.457711] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:16 2020][8592500.463034] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:16 2020][8592500.468536] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.475524] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.482922] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:16 2020][8592500.488933] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:16 2020][8592500.495296] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:16 2020][8592500.501919] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.508629] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:16 2020][8592500.515772] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:16 2020][8592500.521006] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:16 2020][8592500.526677] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.534076] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.541062] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.548379] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.555261] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.562575] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:16 2020][8592500.569456] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:16 2020][8592500.575674] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.582413] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.589060] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.596111] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:16 2020][8592500.601637] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.608731] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.616483] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:16 2020][8592500.623736] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.631596] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.638560] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:16 2020][8592500.644000] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.650471] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.658042] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:16 2020][8592500.663104] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.669379] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:16 2020][8592500.676015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.682282] ll_ost_io03_032 D ffff9c5980174100 0 68847 2 0x00000080 [Tue Mar 17 18:00:16 2020][8592500.689553] Call Trace: [Tue Mar 17 18:00:16 2020][8592500.692180] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:16 2020][8592500.698794] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:16 2020][8592500.705155] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:16 2020][8592500.710476] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:16 2020][8592500.715980] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.722956] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.730360] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:16 2020][8592500.736391] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:16 2020][8592500.742755] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:16 2020][8592500.749373] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:16 2020][8592500.754869] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:16 2020][8592500.760102] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:16 2020][8592500.767244] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:16 2020][8592500.772479] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:16 2020][8592500.778151] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.785551] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.792521] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.799849] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.806735] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.814047] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:16 2020][8592500.820930] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:16 2020][8592500.827148] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.833858] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:16 2020][8592500.839382] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.846463] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.854212] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:16 2020][8592500.861462] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.869339] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.876306] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:16 2020][8592500.881744] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.888220] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592500.895789] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:16 2020][8592500.900851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.907125] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:16 2020][8592500.913747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592500.920022] ll_ost_io01_035 D ffff9c5980175140 0 68848 2 0x00000080 [Tue Mar 17 18:00:16 2020][8592500.927298] Call Trace: [Tue Mar 17 18:00:16 2020][8592500.929934] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.937598] [] schedule+0x29/0x70 [Tue Mar 17 18:00:16 2020][8592500.942763] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.949817] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:16 2020][8592500.955834] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.963062] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.969772] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.977517] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:16 2020][8592500.983707] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592500.990506] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592500.997845] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.005419] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.012561] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:16 2020][8592501.018836] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:16 2020][8592501.025629] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:16 2020][8592501.031846] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.038584] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.045234] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.052303] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:16 2020][8592501.059615] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:16 2020][8592501.065387] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:16 2020][8592501.070914] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.077997] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.085749] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:16 2020][8592501.093004] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.100862] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.107828] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:16 2020][8592501.113264] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.119739] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.127311] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:16 2020][8592501.132380] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592501.138647] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:16 2020][8592501.145269] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592501.151546] ll_ost_io02_041 D ffff9c5980176180 0 68849 2 0x00000080 [Tue Mar 17 18:00:16 2020][8592501.158818] Call Trace: [Tue Mar 17 18:00:16 2020][8592501.161450] [] schedule+0x29/0x70 [Tue Mar 17 18:00:16 2020][8592501.166603] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:16 2020][8592501.173653] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:16 2020][8592501.179670] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592501.186896] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:16 2020][8592501.192755] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:16 2020][8592501.199470] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.206785] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:16 2020][8592501.212979] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:16 2020][8592501.219773] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.227092] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.234675] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.241820] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:16 2020][8592501.248617] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:16 2020][8592501.254834] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.261559] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:16 2020][8592501.267516] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.274165] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.281218] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:16 2020][8592501.286988] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:16 2020][8592501.292515] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.299604] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.307359] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:16 2020][8592501.314610] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.322472] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.329452] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:16 2020][8592501.334882] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.341357] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:16 2020][8592501.348928] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:16 2020][8592501.353989] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592501.360267] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:16 2020][8592501.366904] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:16 2020][8592501.373176] ll_ost_io02_042 D ffff9c597d7e0000 0 68850 2 0x00000080 [Tue Mar 17 18:00:16 2020][8592501.380444] Call Trace: [Tue Mar 17 18:00:16 2020][8592501.383074] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:16 2020][8592501.389699] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:16 2020][8592501.396074] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:16 2020][8592501.401388] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:16 2020][8592501.406891] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.413867] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:16 2020][8592501.421270] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:16 2020][8592501.427290] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:16 2020][8592501.433652] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:17 2020][8592501.440272] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.446978] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:17 2020][8592501.454130] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:17 2020][8592501.459359] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:17 2020][8592501.465037] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.472442] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:17 2020][8592501.479413] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.486736] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.493624] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.500940] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:17 2020][8592501.507820] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:17 2020][8592501.514037] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.520790] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.527440] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.534493] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:17 2020][8592501.540532] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.547967] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.555053] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.562804] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:17 2020][8592501.570065] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.577925] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.584890] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:17 2020][8592501.590344] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.596817] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.604390] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:17 2020][8592501.609450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.615725] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:17 2020][8592501.622346] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.628622] ll_ost_io01_036 D ffff9c597d7e1040 0 68851 2 0x00000080 [Tue Mar 17 18:00:17 2020][8592501.635896] Call Trace: [Tue Mar 17 18:00:17 2020][8592501.638525] [] schedule+0x29/0x70 [Tue Mar 17 18:00:17 2020][8592501.643673] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.650752] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.657888] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:17 2020][8592501.663908] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:17 2020][8592501.670094] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:17 2020][8592501.676892] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:17 2020][8592501.683107] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.689846] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.696496] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.703546] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:17 2020][8592501.709072] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.716177] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.723924] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:17 2020][8592501.731179] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.739038] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.746004] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:17 2020][8592501.751440] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.757915] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.765486] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:17 2020][8592501.770547] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.776823] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:17 2020][8592501.783461] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.789736] ll_ost_io02_043 D ffff9c597d7e2080 0 68852 2 0x00000080 [Tue Mar 17 18:00:17 2020][8592501.797012] Call Trace: [Tue Mar 17 18:00:17 2020][8592501.799642] [] schedule+0x29/0x70 [Tue Mar 17 18:00:17 2020][8592501.804796] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.811860] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592501.819005] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:17 2020][8592501.825022] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:17 2020][8592501.831211] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:17 2020][8592501.838005] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:17 2020][8592501.844236] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.850971] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.857626] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.864697] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:17 2020][8592501.872011] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:17 2020][8592501.877765] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:17 2020][8592501.883289] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.890370] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.898117] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:17 2020][8592501.905372] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.913246] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.920212] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:17 2020][8592501.925643] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.932115] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592501.939689] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:17 2020][8592501.944750] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.951024] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:17 2020][8592501.957646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592501.963920] ll_ost_io01_037 D ffff9c597d7e30c0 0 68853 2 0x00000080 [Tue Mar 17 18:00:17 2020][8592501.971195] Call Trace: [Tue Mar 17 18:00:17 2020][8592501.973825] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:17 2020][8592501.980448] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:17 2020][8592501.986823] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:17 2020][8592501.992138] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:17 2020][8592501.997641] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.004616] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.012013] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:17 2020][8592502.018029] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:17 2020][8592502.024393] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:17 2020][8592502.031009] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:17 2020][8592502.036519] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:17 2020][8592502.041747] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:17 2020][8592502.048888] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:17 2020][8592502.054123] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:17 2020][8592502.059797] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.067197] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:17 2020][8592502.074166] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.081481] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.088361] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.095675] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:17 2020][8592502.102572] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:17 2020][8592502.108791] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.115532] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.122177] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.129227] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:17 2020][8592502.134749] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.141836] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.149583] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:17 2020][8592502.156836] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.164695] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.171676] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:17 2020][8592502.177107] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.183581] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.191153] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:17 2020][8592502.196215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592502.202489] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:17 2020][8592502.209110] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:17 2020][8592502.215386] ll_ost_io00_038 D ffff9c597d7e4100 0 68854 2 0x00000080 [Tue Mar 17 18:00:17 2020][8592502.222661] Call Trace: [Tue Mar 17 18:00:17 2020][8592502.225299] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.232961] [] schedule+0x29/0x70 [Tue Mar 17 18:00:17 2020][8592502.238126] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:17 2020][8592502.245180] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:17 2020][8592502.251190] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:17 2020][8592502.258417] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:17 2020][8592502.265129] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.272873] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:17 2020][8592502.279061] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:17 2020][8592502.285861] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.293199] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.300788] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:17 2020][8592502.307932] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:17 2020][8592502.314206] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:17 2020][8592502.321000] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:17 2020][8592502.327219] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.333928] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:17 2020][8592502.339886] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.346538] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.353612] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:17 2020][8592502.360927] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:17 2020][8592502.366695] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:17 2020][8592502.372216] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.379304] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.387052] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:17 2020][8592502.394304] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.402165] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.409131] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:17 2020][8592502.414567] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.421039] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:17 2020][8592502.428612] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:17 2020][8592502.433689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.439957] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:18 2020][8592502.446580] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.452854] ll_ost_io02_044 D ffff9c597d7e5140 0 68855 2 0x00000080 [Tue Mar 17 18:00:18 2020][8592502.460137] Call Trace: [Tue Mar 17 18:00:18 2020][8592502.462767] [] schedule+0x29/0x70 [Tue Mar 17 18:00:18 2020][8592502.467911] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.474962] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:18 2020][8592502.480982] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.488205] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:18 2020][8592502.494062] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.500781] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.508097] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:18 2020][8592502.514287] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.521082] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.528399] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.535985] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.543130] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:18 2020][8592502.549924] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:18 2020][8592502.556143] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.562897] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.569545] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.576614] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:18 2020][8592502.583931] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:18 2020][8592502.589685] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:18 2020][8592502.595210] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.602299] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.610045] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:18 2020][8592502.617299] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.625160] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.632140] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:18 2020][8592502.637568] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.644043] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.651617] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:18 2020][8592502.656678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.662953] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:18 2020][8592502.669575] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.675849] ll_ost_io01_038 D ffff9c597d7e6180 0 68856 2 0x00000080 [Tue Mar 17 18:00:18 2020][8592502.683124] Call Trace: [Tue Mar 17 18:00:18 2020][8592502.685753] [] schedule+0x29/0x70 [Tue Mar 17 18:00:18 2020][8592502.690905] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.697980] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:18 2020][8592502.703992] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.711221] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.717932] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.725682] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:18 2020][8592502.731874] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.738670] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.745987] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.753569] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.760731] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:18 2020][8592502.767008] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:18 2020][8592502.773544] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:18 2020][8592502.779846] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.786934] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.794681] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:18 2020][8592502.801934] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.809795] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.816761] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:18 2020][8592502.822196] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.828687] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.836259] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:18 2020][8592502.841316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.847588] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:18 2020][8592502.854209] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592502.860483] ll_ost_io01_039 D ffff9c5980360000 0 68857 2 0x00000080 [Tue Mar 17 18:00:18 2020][8592502.867752] Call Trace: [Tue Mar 17 18:00:18 2020][8592502.870380] [] schedule+0x29/0x70 [Tue Mar 17 18:00:18 2020][8592502.875523] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.882576] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:18 2020][8592502.888609] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.895835] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:18 2020][8592502.901678] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.908394] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.915707] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:18 2020][8592502.921900] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:18 2020][8592502.928697] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.936013] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.943595] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592502.950735] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:18 2020][8592502.957544] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:18 2020][8592502.963764] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.970504] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.977151] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592502.984217] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:18 2020][8592502.991533] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:18 2020][8592502.997288] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:18 2020][8592503.002815] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.009894] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.017643] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:18 2020][8592503.024910] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.032772] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.039739] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:18 2020][8592503.045177] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.051649] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.059222] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:18 2020][8592503.064284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592503.070557] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:18 2020][8592503.077179] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592503.083454] ll_ost_io00_039 D ffff9c5980361040 0 68858 2 0x00000080 [Tue Mar 17 18:00:18 2020][8592503.090745] Call Trace: [Tue Mar 17 18:00:18 2020][8592503.093376] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:18 2020][8592503.098698] [] schedule+0x29/0x70 [Tue Mar 17 18:00:18 2020][8592503.103846] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:18 2020][8592503.109864] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 18:00:18 2020][8592503.116312] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:18 2020][8592503.122151] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:18 2020][8592503.127472] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:18 2020][8592503.133661] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:18 2020][8592503.139068] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:18 2020][8592503.144490] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:00:18 2020][8592503.150498] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:18 2020][8592503.155821] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:00:18 2020][8592503.162703] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592503.168804] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:00:18 2020][8592503.174387] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:00:18 2020][8592503.181273] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.188071] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:18 2020][8592503.195652] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.203931] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.211787] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.218421] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:00:18 2020][8592503.225247] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.233205] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.240378] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.247083] [] ? generic_setxattr+0x68/0x80 [Tue Mar 17 18:00:18 2020][8592503.253115] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:00:18 2020][8592503.259914] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.266981] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.274123] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:18 2020][8592503.280329] [] ofd_object_punch+0x4bb/0xd30 [ofd] [Tue Mar 17 18:00:18 2020][8592503.286865] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:18 2020][8592503.293183] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.300263] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.308028] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:18 2020][8592503.315284] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.323143] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.330109] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:18 2020][8592503.335545] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.342020] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:18 2020][8592503.349607] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:18 2020][8592503.354668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592503.360936] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:18 2020][8592503.367557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:18 2020][8592503.373833] ll_ost_io02_045 D ffff9c5980362080 0 68859 2 0x00000080 [Tue Mar 17 18:00:18 2020][8592503.381107] Call Trace: [Tue Mar 17 18:00:18 2020][8592503.383736] [] schedule+0x29/0x70 [Tue Mar 17 18:00:18 2020][8592503.388885] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.395948] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:18 2020][8592503.403091] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:18 2020][8592503.409125] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:18 2020][8592503.415313] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:19 2020][8592503.422110] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:19 2020][8592503.428326] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.435065] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.441714] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.448782] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:19 2020][8592503.456096] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:19 2020][8592503.461851] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:19 2020][8592503.467378] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.474480] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.482232] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:19 2020][8592503.489484] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.497345] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.504310] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:19 2020][8592503.509745] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.516220] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.523792] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:19 2020][8592503.528853] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592503.535128] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:19 2020][8592503.541764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592503.548032] ll_ost_io02_046 D ffff9c59803630c0 0 68860 2 0x00000080 [Tue Mar 17 18:00:19 2020][8592503.555300] Call Trace: [Tue Mar 17 18:00:19 2020][8592503.557929] [] schedule+0x29/0x70 [Tue Mar 17 18:00:19 2020][8592503.563074] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.570125] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:19 2020][8592503.576140] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.583371] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.590080] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.597823] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:19 2020][8592503.604028] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.610829] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.618146] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.625729] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.632877] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:19 2020][8592503.639150] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:19 2020][8592503.645945] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:19 2020][8592503.652161] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.658900] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.665551] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.672633] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:19 2020][8592503.679941] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:19 2020][8592503.685695] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:19 2020][8592503.692256] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.699688] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.706768] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.714517] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:19 2020][8592503.721770] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.729629] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.736597] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:19 2020][8592503.742045] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.748521] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.756097] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:19 2020][8592503.761157] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592503.767431] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:19 2020][8592503.774052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592503.780326] ll_ost_io01_040 D ffff9c5980364100 0 68861 2 0x00000080 [Tue Mar 17 18:00:19 2020][8592503.787593] Call Trace: [Tue Mar 17 18:00:19 2020][8592503.790233] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.797897] [] schedule+0x29/0x70 [Tue Mar 17 18:00:19 2020][8592503.803062] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.810115] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:19 2020][8592503.816131] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.823361] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.830071] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.837814] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:19 2020][8592503.844005] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:19 2020][8592503.850802] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.858118] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.865700] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592503.872863] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:19 2020][8592503.879140] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:19 2020][8592503.885937] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:19 2020][8592503.892152] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.898892] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.905540] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.912609] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:19 2020][8592503.919922] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:19 2020][8592503.925677] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:19 2020][8592503.931203] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.938299] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.946049] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:19 2020][8592503.953301] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.961163] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.968126] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:19 2020][8592503.973563] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.980036] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592503.987610] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:19 2020][8592503.992669] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592503.998946] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:19 2020][8592504.005583] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592504.011858] ll_ost_io02_047 D ffff9c5980365140 0 68862 2 0x00000080 [Tue Mar 17 18:00:19 2020][8592504.019125] Call Trace: [Tue Mar 17 18:00:19 2020][8592504.021755] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:19 2020][8592504.027068] [] schedule+0x29/0x70 [Tue Mar 17 18:00:19 2020][8592504.032216] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:19 2020][8592504.038232] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 18:00:19 2020][8592504.044680] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:19 2020][8592504.050520] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:19 2020][8592504.055843] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:19 2020][8592504.062046] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:19 2020][8592504.067446] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:19 2020][8592504.072854] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:00:19 2020][8592504.078435] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:19 2020][8592504.083759] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:00:19 2020][8592504.090205] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592504.096311] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:00:19 2020][8592504.103197] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.109994] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:19 2020][8592504.117574] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.125867] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.133702] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.140344] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:00:19 2020][8592504.147173] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.155125] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.162292] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.168994] [] ? generic_setxattr+0x68/0x80 [Tue Mar 17 18:00:19 2020][8592504.175029] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:00:19 2020][8592504.181828] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.188892] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.196055] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:19 2020][8592504.202245] [] ofd_object_punch+0x4bb/0xd30 [ofd] [Tue Mar 17 18:00:19 2020][8592504.208779] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:19 2020][8592504.215083] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.222171] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.229918] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:19 2020][8592504.237170] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.245033] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.251998] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:19 2020][8592504.257433] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.263922] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:19 2020][8592504.271496] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:19 2020][8592504.276559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592504.282835] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:19 2020][8592504.289455] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:19 2020][8592504.295744] ll_ost_io01_041 D ffff9c5980366180 0 68863 2 0x00000080 [Tue Mar 17 18:00:19 2020][8592504.303012] Call Trace: [Tue Mar 17 18:00:19 2020][8592504.305642] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:19 2020][8592504.312278] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:19 2020][8592504.318635] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:19 2020][8592504.323958] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:19 2020][8592504.329472] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.336442] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.343839] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:19 2020][8592504.349857] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:19 2020][8592504.356218] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:19 2020][8592504.362839] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.369544] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:19 2020][8592504.376683] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:19 2020][8592504.381909] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:19 2020][8592504.387599] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.395001] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:19 2020][8592504.401971] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.409287] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:19 2020][8592504.416175] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.423488] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:20 2020][8592504.430369] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:20 2020][8592504.436587] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.443327] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.449976] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.457046] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:20 2020][8592504.462568] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.469648] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.477397] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:20 2020][8592504.484649] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.492511] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.499474] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:20 2020][8592504.504912] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.511386] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.518957] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:20 2020][8592504.524035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.530309] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:20 2020][8592504.536923] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.543199] ll_ost_io02_048 D ffff9c59178a8000 0 68864 2 0x00000080 [Tue Mar 17 18:00:20 2020][8592504.550475] Call Trace: [Tue Mar 17 18:00:20 2020][8592504.553103] [] schedule+0x29/0x70 [Tue Mar 17 18:00:20 2020][8592504.558255] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.565308] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:20 2020][8592504.571324] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.578553] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.585278] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.593023] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:20 2020][8592504.599215] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.606012] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.613336] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.620919] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.628058] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:20 2020][8592504.634333] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:20 2020][8592504.640868] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:20 2020][8592504.647171] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.654273] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.662023] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:20 2020][8592504.669277] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.677139] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.684102] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:20 2020][8592504.689540] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.696014] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.703586] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:20 2020][8592504.708646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.714923] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:20 2020][8592504.721556] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.727825] ll_ost_io01_042 D ffff9c59178a9040 0 68865 2 0x00000080 [Tue Mar 17 18:00:20 2020][8592504.735101] Call Trace: [Tue Mar 17 18:00:20 2020][8592504.737731] [] schedule+0x29/0x70 [Tue Mar 17 18:00:20 2020][8592504.742876] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.749927] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:20 2020][8592504.755944] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.763171] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.769882] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.777625] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:20 2020][8592504.783830] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.790631] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.797948] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.805530] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.812677] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:20 2020][8592504.818952] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:20 2020][8592504.825747] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:20 2020][8592504.831962] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.838666] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:20 2020][8592504.844704] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.852155] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.859234] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.866987] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:20 2020][8592504.874243] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.882105] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.889072] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:20 2020][8592504.894507] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.900983] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592504.908550] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:20 2020][8592504.913606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.919880] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:20 2020][8592504.926516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592504.932784] ll_ost_io01_043 D ffff9c59178aa080 0 68866 2 0x00000080 [Tue Mar 17 18:00:20 2020][8592504.940069] Call Trace: [Tue Mar 17 18:00:20 2020][8592504.942700] [] schedule+0x29/0x70 [Tue Mar 17 18:00:20 2020][8592504.947851] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.954903] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:20 2020][8592504.960920] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.968148] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.974858] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592504.982617] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:20 2020][8592504.988801] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:20 2020][8592504.995599] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.002915] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.010497] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.017636] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:20 2020][8592505.023910] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:20 2020][8592505.030446] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:20 2020][8592505.036750] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592505.043837] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:20 2020][8592505.051599] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:20 2020][8592505.058026] INFO: task kswapd0:276 blocked for more than 120 seconds. [Tue Mar 17 18:00:20 2020][8592505.058027] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:20 2020][8592505.058031] kswapd0 D ffff9c5a3a11c100 0 276 2 0x00000000 [Tue Mar 17 18:00:20 2020][8592505.058032] Call Trace: [Tue Mar 17 18:00:20 2020][8592505.058064] [] ? start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:20 2020][8592505.058070] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:20 2020][8592505.058073] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:20 2020][8592505.058075] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:20 2020][8592505.058079] [] dquot_release+0x72/0x110 [Tue Mar 17 18:00:20 2020][8592505.058091] [] ldiskfs_release_dquot+0x6f/0xa0 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058093] [] dqput+0xbd/0x1f0 [Tue Mar 17 18:00:20 2020][8592505.058095] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 18:00:20 2020][8592505.058096] [] dquot_drop+0x35/0x40 [Tue Mar 17 18:00:20 2020][8592505.058105] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058113] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058116] [] evict+0xb4/0x180 [Tue Mar 17 18:00:20 2020][8592505.058118] [] dispose_list+0x3e/0x50 [Tue Mar 17 18:00:20 2020][8592505.058119] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 18:00:20 2020][8592505.058121] [] prune_super+0x173/0x190 [Tue Mar 17 18:00:20 2020][8592505.058126] [] shrink_slab+0x175/0x340 [Tue Mar 17 18:00:20 2020][8592505.058129] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 18:00:20 2020][8592505.058131] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 18:00:20 2020][8592505.058132] [] kswapd+0x173/0x440 [Tue Mar 17 18:00:20 2020][8592505.058136] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:20 2020][8592505.058138] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 18:00:20 2020][8592505.058139] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:20 2020][8592505.058140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592505.058143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:20 2020][8592505.058144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592505.058146] INFO: task kswapd1:277 blocked for more than 120 seconds. [Tue Mar 17 18:00:20 2020][8592505.058146] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:20 2020][8592505.058148] kswapd1 D ffff9c5a3a11b0c0 0 277 2 0x00000000 [Tue Mar 17 18:00:20 2020][8592505.058149] Call Trace: [Tue Mar 17 18:00:20 2020][8592505.058153] [] ? start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:20 2020][8592505.058155] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:20 2020][8592505.058156] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:20 2020][8592505.058157] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:20 2020][8592505.058159] [] dquot_release+0x72/0x110 [Tue Mar 17 18:00:20 2020][8592505.058167] [] ldiskfs_release_dquot+0x6f/0xa0 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058168] [] dqput+0xbd/0x1f0 [Tue Mar 17 18:00:20 2020][8592505.058170] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 18:00:20 2020][8592505.058171] [] dquot_drop+0x35/0x40 [Tue Mar 17 18:00:20 2020][8592505.058179] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058186] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 18:00:20 2020][8592505.058187] [] evict+0xb4/0x180 [Tue Mar 17 18:00:20 2020][8592505.058188] [] dispose_list+0x3e/0x50 [Tue Mar 17 18:00:20 2020][8592505.058190] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 18:00:20 2020][8592505.058191] [] prune_super+0x173/0x190 [Tue Mar 17 18:00:20 2020][8592505.058192] [] shrink_slab+0x175/0x340 [Tue Mar 17 18:00:20 2020][8592505.058194] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 18:00:20 2020][8592505.058195] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 18:00:20 2020][8592505.058197] [] kswapd+0x173/0x440 [Tue Mar 17 18:00:20 2020][8592505.058198] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:20 2020][8592505.058200] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 18:00:20 2020][8592505.058201] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:20 2020][8592505.058202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592505.058204] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:20 2020][8592505.058205] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:20 2020][8592505.058206] INFO: task kswapd2:278 blocked for more than 120 seconds. [Tue Mar 17 18:00:20 2020][8592505.058207] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058209] kswapd2 D ffff9c5a3a250000 0 278 2 0x00000000 [Tue Mar 17 18:00:21 2020][8592505.058209] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058212] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 18:00:21 2020][8592505.058213] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058217] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058219] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058223] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058227] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058231] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:21 2020][8592505.058235] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058243] [] ? ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058251] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058258] [] ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058259] [] dqput+0xbd/0x1f0 [Tue Mar 17 18:00:21 2020][8592505.058261] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 18:00:21 2020][8592505.058262] [] dquot_drop+0x35/0x40 [Tue Mar 17 18:00:21 2020][8592505.058269] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058276] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058277] [] evict+0xb4/0x180 [Tue Mar 17 18:00:21 2020][8592505.058279] [] dispose_list+0x3e/0x50 [Tue Mar 17 18:00:21 2020][8592505.058280] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 18:00:21 2020][8592505.058281] [] prune_super+0x173/0x190 [Tue Mar 17 18:00:21 2020][8592505.058283] [] shrink_slab+0x175/0x340 [Tue Mar 17 18:00:21 2020][8592505.058284] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058286] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 18:00:21 2020][8592505.058287] [] kswapd+0x173/0x440 [Tue Mar 17 18:00:21 2020][8592505.058289] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058290] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 18:00:21 2020][8592505.058291] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058294] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058297] INFO: task kswapd3:279 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058297] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058299] kswapd3 D ffff9c5a3a251040 0 279 2 0x00000000 [Tue Mar 17 18:00:21 2020][8592505.058299] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058300] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 18:00:21 2020][8592505.058302] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058306] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058307] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058311] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058315] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058318] [] ? unfreeze_partials.isra.42+0xd2/0x130 [Tue Mar 17 18:00:21 2020][8592505.058319] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:21 2020][8592505.058323] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:21 2020][8592505.058330] [] ? ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058337] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058344] [] ldiskfs_release_dquot+0x5c/0xa0 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058345] [] dqput+0xbd/0x1f0 [Tue Mar 17 18:00:21 2020][8592505.058347] [] __dquot_drop+0x85/0xc0 [Tue Mar 17 18:00:21 2020][8592505.058348] [] dquot_drop+0x35/0x40 [Tue Mar 17 18:00:21 2020][8592505.058355] [] ldiskfs_clear_inode+0x22/0x90 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058362] [] ldiskfs_evict_inode+0x9f/0x630 [ldiskfs] [Tue Mar 17 18:00:21 2020][8592505.058363] [] evict+0xb4/0x180 [Tue Mar 17 18:00:21 2020][8592505.058364] [] dispose_list+0x3e/0x50 [Tue Mar 17 18:00:21 2020][8592505.058366] [] prune_icache_sb+0x17c/0x330 [Tue Mar 17 18:00:21 2020][8592505.058367] [] prune_super+0x173/0x190 [Tue Mar 17 18:00:21 2020][8592505.058368] [] shrink_slab+0x175/0x340 [Tue Mar 17 18:00:21 2020][8592505.058370] [] ? compaction_suitable+0x5b/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058371] [] balance_pgdat+0x4b1/0x5e0 [Tue Mar 17 18:00:21 2020][8592505.058373] [] kswapd+0x173/0x440 [Tue Mar 17 18:00:21 2020][8592505.058374] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058375] [] ? balance_pgdat+0x5e0/0x5e0 [Tue Mar 17 18:00:21 2020][8592505.058377] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058378] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058379] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058381] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058513] INFO: task md6_raid6:65600 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058513] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058515] md6_raid6 D ffff9c5a162c30c0 0 65600 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058515] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058517] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058520] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058521] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058525] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058526] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:21 2020][8592505.058531] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 18:00:21 2020][8592505.058532] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 18:00:21 2020][8592505.058535] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:00:21 2020][8592505.058537] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:21 2020][8592505.058538] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058539] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:21 2020][8592505.058541] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058543] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058546] INFO: task md10_raid6:65692 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058546] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058548] md10_raid6 D ffff9c6a37fe4100 0 65692 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058548] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058550] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058551] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058553] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058554] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058555] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:21 2020][8592505.058558] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 18:00:21 2020][8592505.058560] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 18:00:21 2020][8592505.058561] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:00:21 2020][8592505.058562] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:21 2020][8592505.058564] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058565] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:21 2020][8592505.058566] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058567] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058569] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058572] INFO: task md4_raid6:65845 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058572] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058574] md4_raid6 D ffff9c8a04e94100 0 65845 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058574] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058576] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058577] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058578] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058580] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058581] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:21 2020][8592505.058584] [] raid5d+0x48a/0x670 [raid456] [Tue Mar 17 18:00:21 2020][8592505.058585] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:21 2020][8592505.058587] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058588] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:21 2020][8592505.058589] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058590] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058592] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058595] INFO: task md8_raid6:66071 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058595] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058596] md8_raid6 D ffff9c6a02684100 0 66071 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058597] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058599] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058600] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058601] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058603] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058604] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:21 2020][8592505.058606] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 18:00:21 2020][8592505.058608] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 18:00:21 2020][8592505.058609] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:00:21 2020][8592505.058610] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:21 2020][8592505.058612] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058613] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:21 2020][8592505.058614] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058617] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058618] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058620] INFO: task md0_raid6:66171 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058620] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058621] md0_raid6 D ffff9c5a1139c100 0 66171 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058622] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058623] [] schedule+0x29/0x70 [Tue Mar 17 18:00:21 2020][8592505.058625] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:21 2020][8592505.058626] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058628] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058629] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:21 2020][8592505.058631] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 18:00:21 2020][8592505.058633] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 18:00:21 2020][8592505.058634] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:00:21 2020][8592505.058635] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:21 2020][8592505.058637] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:21 2020][8592505.058638] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:21 2020][8592505.058639] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:21 2020][8592505.058640] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058642] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:21 2020][8592505.058643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:21 2020][8592505.058646] INFO: task md2_raid6:66282 blocked for more than 120 seconds. [Tue Mar 17 18:00:21 2020][8592505.058646] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [Tue Mar 17 18:00:21 2020][8592505.058648] md2_raid6 D ffff9c5a3a3d9040 0 66282 2 0x00000080 [Tue Mar 17 18:00:21 2020][8592505.058648] Call Trace: [Tue Mar 17 18:00:21 2020][8592505.058650] [] schedule+0x29/0x70 [Tue Mar 17 18:00:22 2020][8592505.058651] [] percpu_ref_switch_to_atomic_sync+0x65/0xb0 [Tue Mar 17 18:00:22 2020][8592505.058652] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592505.058654] [] set_in_sync+0x67/0xe0 [Tue Mar 17 18:00:22 2020][8592505.058655] [] md_check_recovery+0x27f/0x500 [Tue Mar 17 18:00:22 2020][8592505.058657] [] raid5d+0x56/0x670 [raid456] [Tue Mar 17 18:00:22 2020][8592505.058659] [] ? schedule_timeout+0x170/0x2d0 [Tue Mar 17 18:00:22 2020][8592505.058660] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:00:22 2020][8592505.058661] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:00:22 2020][8592505.058663] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592505.058664] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:00:22 2020][8592505.058665] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592505.058666] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592505.058668] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592505.058669] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.501131] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.508991] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.515961] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:22 2020][8592506.521391] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.527864] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.535437] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592506.540499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.546773] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592506.553394] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.559669] ll_ost_io01_044 D ffff9c59178ab0c0 0 68867 2 0x00000080 [Tue Mar 17 18:00:22 2020][8592506.566963] Call Trace: [Tue Mar 17 18:00:22 2020][8592506.569599] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.577264] [] schedule+0x29/0x70 [Tue Mar 17 18:00:22 2020][8592506.582414] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.589466] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592506.595483] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.602711] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.609420] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.617164] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:22 2020][8592506.623355] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.630168] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.637483] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.645058] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.652199] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:22 2020][8592506.658474] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:22 2020][8592506.665268] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:22 2020][8592506.671484] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.678195] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:22 2020][8592506.684153] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.690801] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.697867] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:22 2020][8592506.703387] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.710474] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.718223] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:22 2020][8592506.725475] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.733334] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.740302] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:22 2020][8592506.745737] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.752210] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.759784] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592506.764861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.771127] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592506.777750] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.784023] ll_ost_io01_045 D ffff9c59178ac100 0 68868 2 0x00000080 [Tue Mar 17 18:00:22 2020][8592506.791292] Call Trace: [Tue Mar 17 18:00:22 2020][8592506.793929] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.801593] [] schedule+0x29/0x70 [Tue Mar 17 18:00:22 2020][8592506.806742] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.813794] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592506.819812] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.827054] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.833768] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.841511] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:22 2020][8592506.847702] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592506.854498] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.861816] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.869397] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592506.876536] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:22 2020][8592506.882810] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:22 2020][8592506.889607] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:22 2020][8592506.895839] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.902543] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:22 2020][8592506.908070] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.915158] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.922912] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:22 2020][8592506.930166] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.938025] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.944992] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:22 2020][8592506.950429] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.956903] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592506.964488] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592506.969544] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.975819] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592506.982439] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592506.988706] ll_ost_io03_033 D ffff9c59178ad140 0 68869 2 0x00000080 [Tue Mar 17 18:00:22 2020][8592506.995987] Call Trace: [Tue Mar 17 18:00:22 2020][8592506.998620] [] schedule+0x29/0x70 [Tue Mar 17 18:00:22 2020][8592507.003772] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.010825] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592507.016840] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.024083] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:22 2020][8592507.029927] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.036643] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.043956] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:22 2020][8592507.050149] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.056946] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.064270] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.071852] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.078994] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:22 2020][8592507.085795] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:22 2020][8592507.092030] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.098740] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:22 2020][8592507.104697] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.111346] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.118395] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:22 2020][8592507.124439] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.131870] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.138957] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.146705] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:22 2020][8592507.153958] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.161832] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.168801] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:22 2020][8592507.174239] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.180712] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.188284] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592507.193344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592507.199622] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592507.206240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592507.212517] ll_ost_io00_040 D ffff9c59178ae180 0 68870 2 0x00000080 [Tue Mar 17 18:00:22 2020][8592507.219799] Call Trace: [Tue Mar 17 18:00:22 2020][8592507.222430] [] schedule+0x29/0x70 [Tue Mar 17 18:00:22 2020][8592507.227583] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.234648] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:22 2020][8592507.240658] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.247886] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:22 2020][8592507.253729] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.260443] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.267760] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:22 2020][8592507.273947] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:22 2020][8592507.280746] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.288089] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.295664] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:22 2020][8592507.302829] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:22 2020][8592507.309620] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:22 2020][8592507.315840] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.322549] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:22 2020][8592507.328075] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.335154] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.342919] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:22 2020][8592507.350173] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.358033] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.365014] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:22 2020][8592507.370442] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.376918] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:22 2020][8592507.384490] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:22 2020][8592507.389550] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592507.395827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:22 2020][8592507.402448] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:22 2020][8592507.408723] ll_ost_io03_034 D ffff9c597fdf0000 0 68871 2 0x00000080 [Tue Mar 17 18:00:22 2020][8592507.415999] Call Trace: [Tue Mar 17 18:00:23 2020][8592507.418628] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592507.423779] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.430846] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:23 2020][8592507.436855] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.444083] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:23 2020][8592507.449926] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.456639] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.463954] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:23 2020][8592507.470144] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.476942] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.484272] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.491850] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.498995] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:23 2020][8592507.505793] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:23 2020][8592507.512011] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.518749] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.525398] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.532448] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:23 2020][8592507.537975] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.545055] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.552816] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:23 2020][8592507.560072] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.567932] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.574898] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:23 2020][8592507.580335] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.586807] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.594381] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:23 2020][8592507.599443] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592507.605716] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:23 2020][8592507.612337] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592507.618627] ll_ost_io03_035 D ffff9c597fdf1040 0 68872 2 0x00000080 [Tue Mar 17 18:00:23 2020][8592507.625896] Call Trace: [Tue Mar 17 18:00:23 2020][8592507.628534] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.636198] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592507.641350] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.648400] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:23 2020][8592507.654418] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.661645] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.668356] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.676101] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:23 2020][8592507.682305] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.689104] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.696422] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.704002] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.711144] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:23 2020][8592507.717418] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:23 2020][8592507.724211] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:23 2020][8592507.730428] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.737158] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.743808] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.750893] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:23 2020][8592507.758208] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:23 2020][8592507.763963] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:23 2020][8592507.770522] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.777956] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.785035] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.792783] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:23 2020][8592507.800037] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.807898] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.814862] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:23 2020][8592507.820313] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.826791] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.834363] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:23 2020][8592507.839424] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592507.845698] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:23 2020][8592507.852320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592507.858594] ll_ost_io01_046 D ffff9c597fdf2080 0 68873 2 0x00000080 [Tue Mar 17 18:00:23 2020][8592507.865869] Call Trace: [Tue Mar 17 18:00:23 2020][8592507.868499] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592507.873643] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.880710] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:23 2020][8592507.886721] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.893947] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:23 2020][8592507.899789] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.906504] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.913819] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:23 2020][8592507.920011] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592507.926806] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.934123] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.941706] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592507.948859] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:23 2020][8592507.955657] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:23 2020][8592507.961874] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.968614] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.975261] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.982313] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:23 2020][8592507.987839] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592507.994928] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.002673] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:23 2020][8592508.009928] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.017803] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.024770] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:23 2020][8592508.030208] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.036682] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.044252] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:23 2020][8592508.049315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.055590] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:23 2020][8592508.062209] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.068485] ll_ost_io00_041 S ffff9c597fdf30c0 0 68874 2 0x00000080 [Tue Mar 17 18:00:23 2020][8592508.075768] Call Trace: [Tue Mar 17 18:00:23 2020][8592508.078400] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:23 2020][8592508.084500] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592508.089685] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.096565] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:23 2020][8592508.102350] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.108823] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.116394] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:23 2020][8592508.121455] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.127731] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:23 2020][8592508.134351] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.140640] ll_ost_io01_047 D ffff9c597fdf4100 0 68875 2 0x00000080 [Tue Mar 17 18:00:23 2020][8592508.147911] Call Trace: [Tue Mar 17 18:00:23 2020][8592508.150550] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592508.158212] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592508.163364] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.170414] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:23 2020][8592508.176433] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.183659] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.190371] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592508.198113] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:23 2020][8592508.204318] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.211119] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592508.218436] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:23 2020][8592508.226018] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:23 2020][8592508.233167] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:23 2020][8592508.239440] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:23 2020][8592508.246234] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:23 2020][8592508.252455] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.259192] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.265840] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.272920] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:23 2020][8592508.280230] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:23 2020][8592508.285986] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:23 2020][8592508.291511] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.298594] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.306365] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:23 2020][8592508.313616] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.321493] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.328461] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:23 2020][8592508.333913] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.340404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:23 2020][8592508.347980] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:23 2020][8592508.353039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.359316] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:23 2020][8592508.365934] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:23 2020][8592508.372203] ll_ost_io03_036 D ffff9c597fdf5140 0 68876 2 0x00000080 [Tue Mar 17 18:00:23 2020][8592508.379479] Call Trace: [Tue Mar 17 18:00:23 2020][8592508.382108] [] schedule+0x29/0x70 [Tue Mar 17 18:00:23 2020][8592508.387260] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.394310] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:23 2020][8592508.400342] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.407572] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:23 2020][8592508.414285] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.422026] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:24 2020][8592508.428216] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.435015] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.442332] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.449914] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.457054] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:24 2020][8592508.463329] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:24 2020][8592508.469877] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:24 2020][8592508.476183] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.483263] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.491008] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:24 2020][8592508.498263] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.506123] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.513089] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:24 2020][8592508.518526] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.524999] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.532568] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:24 2020][8592508.537638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592508.543908] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:24 2020][8592508.550529] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592508.556803] ll_ost_io03_037 D ffff9c597fdf6180 0 68877 2 0x00000080 [Tue Mar 17 18:00:24 2020][8592508.564080] Call Trace: [Tue Mar 17 18:00:24 2020][8592508.566709] [] schedule+0x29/0x70 [Tue Mar 17 18:00:24 2020][8592508.571861] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.578912] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:24 2020][8592508.584928] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.592158] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:24 2020][8592508.598014] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.604729] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.612045] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:24 2020][8592508.618237] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.625035] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.632359] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.639943] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.647089] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:24 2020][8592508.653883] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:24 2020][8592508.660099] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.666845] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.673494] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.680564] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:24 2020][8592508.687878] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:24 2020][8592508.693635] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:24 2020][8592508.699159] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.706239] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.713990] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:24 2020][8592508.721249] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.729109] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.736074] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:24 2020][8592508.741527] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.748004] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.755582] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:24 2020][8592508.760635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592508.766903] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:24 2020][8592508.773524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592508.779798] ll_ost_io01_048 D ffff9c5915680000 0 68878 2 0x00000080 [Tue Mar 17 18:00:24 2020][8592508.787073] Call Trace: [Tue Mar 17 18:00:24 2020][8592508.789704] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:24 2020][8592508.796325] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:24 2020][8592508.802703] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:24 2020][8592508.808026] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:24 2020][8592508.813528] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.820504] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.827907] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:24 2020][8592508.833927] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:24 2020][8592508.840288] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:24 2020][8592508.846905] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:24 2020][8592508.854048] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:24 2020][8592508.859294] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:24 2020][8592508.864962] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.872363] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592508.879333] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.886645] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.893528] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592508.900841] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:24 2020][8592508.907723] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:24 2020][8592508.913939] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.920670] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.927334] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.934403] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:24 2020][8592508.941720] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:24 2020][8592508.947474] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:24 2020][8592508.954034] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.961468] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.968555] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.976303] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:24 2020][8592508.983558] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.991417] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592508.998398] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:24 2020][8592509.003834] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.010310] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.017882] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:24 2020][8592509.022943] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592509.029218] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:24 2020][8592509.035839] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592509.042115] ll_ost_io03_038 D ffff9c5915681040 0 68879 2 0x00000080 [Tue Mar 17 18:00:24 2020][8592509.049391] Call Trace: [Tue Mar 17 18:00:24 2020][8592509.052018] [] schedule+0x29/0x70 [Tue Mar 17 18:00:24 2020][8592509.057163] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:24 2020][8592509.064230] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:24 2020][8592509.070239] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592509.077465] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:24 2020][8592509.083309] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:24 2020][8592509.090022] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.097338] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:24 2020][8592509.103530] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592509.110327] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.117645] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.125241] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.132383] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:24 2020][8592509.139177] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:24 2020][8592509.145394] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.152105] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:24 2020][8592509.157630] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.164720] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.172474] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:24 2020][8592509.179727] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.187588] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.194570] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:24 2020][8592509.200008] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.206480] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.214054] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:24 2020][8592509.219115] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592509.225389] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:24 2020][8592509.232009] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:24 2020][8592509.238276] ll_ost_io00_042 D ffff9c5915682080 0 68880 2 0x00000080 [Tue Mar 17 18:00:24 2020][8592509.245553] Call Trace: [Tue Mar 17 18:00:24 2020][8592509.248183] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:24 2020][8592509.254805] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:24 2020][8592509.261180] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:24 2020][8592509.266495] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:24 2020][8592509.272002] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.278974] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.286378] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:24 2020][8592509.292652] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:00:24 2020][8592509.297807] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.305058] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.312200] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 18:00:24 2020][8592509.319090] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:24 2020][8592509.324688] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.332352] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.340020] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:00:24 2020][8592509.346556] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.353964] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.361022] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.369209] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:24 2020][8592509.376181] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.383415] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:24 2020][8592509.390742] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:24 2020][8592509.397539] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:24 2020][8592509.403758] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.410497] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:24 2020][8592509.417145] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.424196] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:25 2020][8592509.429721] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.436801] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.444549] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:25 2020][8592509.451802] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.459679] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.466646] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:25 2020][8592509.472082] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.478557] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.486128] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:25 2020][8592509.491189] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.497465] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:25 2020][8592509.504085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.510361] ll_ost_io01_049 D ffff9c59156830c0 0 68881 2 0x00000080 [Tue Mar 17 18:00:25 2020][8592509.517636] Call Trace: [Tue Mar 17 18:00:25 2020][8592509.520266] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:25 2020][8592509.526887] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:25 2020][8592509.533264] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:25 2020][8592509.538578] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:25 2020][8592509.544080] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.551056] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.558452] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:25 2020][8592509.564470] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:25 2020][8592509.570833] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:25 2020][8592509.577455] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.584170] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:25 2020][8592509.591307] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:25 2020][8592509.596541] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:25 2020][8592509.602205] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.609607] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592509.616575] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.623890] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.630772] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.638087] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:25 2020][8592509.644966] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:25 2020][8592509.651198] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.657931] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.664580] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.671650] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:25 2020][8592509.678963] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:25 2020][8592509.684720] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:00:25 2020][8592509.690387] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:25 2020][8592509.695912] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.703003] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.710755] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:25 2020][8592509.718030] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.725896] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.732863] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:25 2020][8592509.738644] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.745119] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.752690] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:25 2020][8592509.757751] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.764026] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:25 2020][8592509.770649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.776922] ll_ost_io00_043 D ffff9c5915684100 0 68882 2 0x00000080 [Tue Mar 17 18:00:25 2020][8592509.784216] Call Trace: [Tue Mar 17 18:00:25 2020][8592509.786845] [] schedule+0x29/0x70 [Tue Mar 17 18:00:25 2020][8592509.792000] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:25 2020][8592509.799059] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:25 2020][8592509.805074] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592509.812299] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:25 2020][8592509.818143] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:25 2020][8592509.824859] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.832173] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:25 2020][8592509.838364] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592509.845177] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.852494] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.860067] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.867208] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:25 2020][8592509.874006] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:25 2020][8592509.880227] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.886929] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:25 2020][8592509.892455] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.899537] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.907293] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:25 2020][8592509.914560] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.922422] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.929387] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:25 2020][8592509.934826] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.941298] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592509.948871] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:25 2020][8592509.953930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.960197] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:25 2020][8592509.966820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592509.973095] ll_ost_io03_039 D ffff9c5915685140 0 68883 2 0x00000080 [Tue Mar 17 18:00:25 2020][8592509.980388] Call Trace: [Tue Mar 17 18:00:25 2020][8592509.983025] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592509.990689] [] schedule+0x29/0x70 [Tue Mar 17 18:00:25 2020][8592509.995839] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.002890] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:25 2020][8592510.008908] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.016136] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.022847] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.030600] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:25 2020][8592510.036788] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.043600] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.050919] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.058501] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.065643] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:25 2020][8592510.071917] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:25 2020][8592510.078710] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:25 2020][8592510.084927] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.091669] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.098317] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.105392] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:25 2020][8592510.112722] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:25 2020][8592510.118469] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:25 2020][8592510.123998] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.131083] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.138831] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:25 2020][8592510.146086] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.153945] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.160912] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:25 2020][8592510.166347] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.172822] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.180408] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:25 2020][8592510.185464] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592510.191739] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:25 2020][8592510.198360] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592510.204633] ll_ost_io01_050 D ffff9c5915686180 0 68884 2 0x00000080 [Tue Mar 17 18:00:25 2020][8592510.211902] Call Trace: [Tue Mar 17 18:00:25 2020][8592510.214532] [] schedule+0x29/0x70 [Tue Mar 17 18:00:25 2020][8592510.219686] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.226751] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.233893] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:25 2020][8592510.239928] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:25 2020][8592510.246117] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:25 2020][8592510.252912] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:25 2020][8592510.259128] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.265859] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.272508] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.279557] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:25 2020][8592510.285084] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.292166] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.299914] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:25 2020][8592510.307204] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.315067] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.322035] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:25 2020][8592510.327471] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.333945] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:25 2020][8592510.341517] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:25 2020][8592510.346580] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592510.352852] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:25 2020][8592510.359475] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:25 2020][8592510.365749] ll_ost_io03_040 D ffff9c5914520000 0 68885 2 0x00000080 [Tue Mar 17 18:00:25 2020][8592510.373024] Call Trace: [Tue Mar 17 18:00:25 2020][8592510.375655] [] schedule+0x29/0x70 [Tue Mar 17 18:00:25 2020][8592510.380806] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.387858] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:25 2020][8592510.393874] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.401101] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:25 2020][8592510.406945] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:25 2020][8592510.413657] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:25 2020][8592510.420974] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:26 2020][8592510.427164] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.433975] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.441295] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.448878] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.456025] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:26 2020][8592510.462821] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:26 2020][8592510.469038] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.475777] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.482424] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.489476] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:26 2020][8592510.495003] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.502101] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.509856] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:26 2020][8592510.517109] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.524938] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:26 2020][8592510.530377] [] ? ptlrpc_server_handle_req_in+0x8df/0xd60 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.538322] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.544797] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.552370] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:26 2020][8592510.557431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592510.563706] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:26 2020][8592510.570341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592510.576609] ll_ost_io00_044 D ffff9c5914521040 0 68886 2 0x00000080 [Tue Mar 17 18:00:26 2020][8592510.583878] Call Trace: [Tue Mar 17 18:00:26 2020][8592510.586514] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.594178] [] schedule+0x29/0x70 [Tue Mar 17 18:00:26 2020][8592510.599328] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.606381] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:26 2020][8592510.612398] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.619624] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.626335] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.634095] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:26 2020][8592510.640279] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.647076] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.654392] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.661975] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.669116] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:26 2020][8592510.675390] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:26 2020][8592510.682184] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:26 2020][8592510.688399] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.695104] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:26 2020][8592510.701158] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.708583] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.715666] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.723422] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:26 2020][8592510.730672] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.738534] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.745498] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:26 2020][8592510.750936] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.757408] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.764981] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:26 2020][8592510.770058] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592510.776325] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:26 2020][8592510.782947] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592510.789222] ll_ost_io00_045 D ffff9c5914522080 0 68887 2 0x00000080 [Tue Mar 17 18:00:26 2020][8592510.796499] Call Trace: [Tue Mar 17 18:00:26 2020][8592510.799128] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:26 2020][8592510.805749] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:26 2020][8592510.812112] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:26 2020][8592510.817432] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:26 2020][8592510.822934] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.829925] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.837323] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:26 2020][8592510.843341] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:26 2020][8592510.849704] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:26 2020][8592510.856322] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:26 2020][8592510.863463] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:26 2020][8592510.868696] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:26 2020][8592510.874368] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.881768] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592510.888739] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.896069] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.902952] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592510.910266] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:26 2020][8592510.917148] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:26 2020][8592510.923363] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.930094] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.936742] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.943812] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:26 2020][8592510.951126] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:26 2020][8592510.956879] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:26 2020][8592510.963450] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.970882] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.977964] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592510.985719] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:26 2020][8592510.992972] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.000833] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.007798] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:26 2020][8592511.013234] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.019709] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.027280] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:26 2020][8592511.032355] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592511.038623] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:26 2020][8592511.045239] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592511.051511] ll_ost_io03_041 D ffff9c59145230c0 0 68888 2 0x00000080 [Tue Mar 17 18:00:26 2020][8592511.058780] Call Trace: [Tue Mar 17 18:00:26 2020][8592511.061418] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.069078] [] schedule+0x29/0x70 [Tue Mar 17 18:00:26 2020][8592511.074223] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:26 2020][8592511.081274] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:26 2020][8592511.087291] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592511.094536] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:26 2020][8592511.101248] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.108991] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:26 2020][8592511.115181] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:26 2020][8592511.121977] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.129294] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.136877] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.144024] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:26 2020][8592511.150298] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:26 2020][8592511.157095] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:26 2020][8592511.163326] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.170055] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.177488] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.184568] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.192318] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:26 2020][8592511.199580] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.207439] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.214405] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:26 2020][8592511.219840] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.226314] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.233903] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:26 2020][8592511.238964] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592511.245240] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:26 2020][8592511.251862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592511.258137] ll_ost_io01_051 D ffff9c5914524100 0 68889 2 0x00000080 [Tue Mar 17 18:00:26 2020][8592511.265413] Call Trace: [Tue Mar 17 18:00:26 2020][8592511.268041] [] schedule+0x29/0x70 [Tue Mar 17 18:00:26 2020][8592511.273195] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.280251] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:26 2020][8592511.287394] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:26 2020][8592511.293429] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:26 2020][8592511.299619] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:26 2020][8592511.306428] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:26 2020][8592511.312647] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.319388] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.326035] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.333102] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:26 2020][8592511.340419] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:26 2020][8592511.346172] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:26 2020][8592511.351700] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.358807] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.366561] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:26 2020][8592511.373814] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.381675] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.388639] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:26 2020][8592511.394077] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.400550] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:26 2020][8592511.408121] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:26 2020][8592511.413183] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:26 2020][8592511.419458] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:27 2020][8592511.426094] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.432363] ll_ost_io03_042 D ffff9c5914525140 0 68890 2 0x00000080 [Tue Mar 17 18:00:27 2020][8592511.439638] Call Trace: [Tue Mar 17 18:00:27 2020][8592511.442268] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:27 2020][8592511.448892] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:27 2020][8592511.455251] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:27 2020][8592511.460574] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:27 2020][8592511.466076] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.473053] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.480455] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:27 2020][8592511.486753] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.493633] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.501992] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.509228] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.516891] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:27 2020][8592511.522477] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.530147] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.538068] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.545469] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.552528] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.560729] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.567700] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.574929] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.582250] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:27 2020][8592511.589044] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:27 2020][8592511.595268] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.601974] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:27 2020][8592511.607725] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:27 2020][8592511.613253] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.620341] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.628106] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:27 2020][8592511.635357] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.643219] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.650186] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:27 2020][8592511.655622] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.662096] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.669667] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:27 2020][8592511.674727] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.681006] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:27 2020][8592511.687624] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.693914] ll_ost_io01_052 S ffff9c5914526180 0 68891 2 0x00000080 [Tue Mar 17 18:00:27 2020][8592511.701182] Call Trace: [Tue Mar 17 18:00:27 2020][8592511.703812] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:27 2020][8592511.709916] [] schedule+0x29/0x70 [Tue Mar 17 18:00:27 2020][8592511.715092] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.721970] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:27 2020][8592511.727754] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.734228] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.741801] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:27 2020][8592511.746861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.753151] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:27 2020][8592511.759765] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.766041] ll_ost_io01_053 D ffff9c5914d50000 0 68892 2 0x00000080 [Tue Mar 17 18:00:27 2020][8592511.773317] Call Trace: [Tue Mar 17 18:00:27 2020][8592511.775945] [] schedule+0x29/0x70 [Tue Mar 17 18:00:27 2020][8592511.781089] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.788140] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:27 2020][8592511.794159] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.801384] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:27 2020][8592511.807228] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.813956] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.821266] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:27 2020][8592511.827456] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.834254] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.841580] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.849161] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592511.856310] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:27 2020][8592511.863105] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:27 2020][8592511.869321] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.876060] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.882724] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.889776] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:27 2020][8592511.895304] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.902391] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.910139] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:27 2020][8592511.917390] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.925252] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.932215] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:27 2020][8592511.937652] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.944127] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592511.951713] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:27 2020][8592511.956768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.963036] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:27 2020][8592511.969657] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592511.975932] ll_ost_io00_046 D ffff9c5914d51040 0 68893 2 0x00000080 [Tue Mar 17 18:00:27 2020][8592511.983209] Call Trace: [Tue Mar 17 18:00:27 2020][8592511.985837] [] schedule+0x29/0x70 [Tue Mar 17 18:00:27 2020][8592511.990988] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:27 2020][8592511.998042] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:27 2020][8592512.004059] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:27 2020][8592512.011298] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:27 2020][8592512.017134] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:27 2020][8592512.023849] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.031164] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:27 2020][8592512.037357] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:27 2020][8592512.044152] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.051470] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.059050] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.066190] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:27 2020][8592512.072986] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:27 2020][8592512.079215] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.085922] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:27 2020][8592512.091961] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.099394] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.106475] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.114223] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:27 2020][8592512.121474] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.129336] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.136301] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:27 2020][8592512.141737] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.148225] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.155800] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:27 2020][8592512.160861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592512.167137] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:27 2020][8592512.173757] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592512.180025] ll_ost_io00_047 D ffff9c5914d52080 0 68894 2 0x00000080 [Tue Mar 17 18:00:27 2020][8592512.187299] Call Trace: [Tue Mar 17 18:00:27 2020][8592512.189927] [] ? wake_up_bit+0x25/0x30 [Tue Mar 17 18:00:27 2020][8592512.195503] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:27 2020][8592512.200824] [] schedule+0x29/0x70 [Tue Mar 17 18:00:27 2020][8592512.205987] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:27 2020][8592512.211997] [] ? qtree_write_dquot+0x126/0x180 [Tue Mar 17 18:00:27 2020][8592512.218267] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:27 2020][8592512.223583] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:27 2020][8592512.229771] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:27 2020][8592512.235179] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:27 2020][8592512.240588] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:00:27 2020][8592512.246603] [] ? __dquot_transfer+0x2e0/0x510 [Tue Mar 17 18:00:27 2020][8592512.252790] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:27 2020][8592512.258111] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:00:27 2020][8592512.265010] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:27 2020][8592512.271114] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:00:27 2020][8592512.276697] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:00:27 2020][8592512.283574] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:00:27 2020][8592512.288988] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:27 2020][8592512.296565] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.304839] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.312515] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.319661] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.327155] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.334139] [] osd_attr_set+0x3f1/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:27 2020][8592512.341020] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:27 2020][8592512.347902] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:27 2020][8592512.354152] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.360885] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.367547] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.374600] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:27 2020][8592512.380129] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.387218] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.394974] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:27 2020][8592512.402249] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.410082] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:27 2020][8592512.415520] [] ? ptlrpc_server_handle_req_in+0x8df/0xd60 [ptlrpc] [Tue Mar 17 18:00:27 2020][8592512.423468] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.429940] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.437513] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:28 2020][8592512.442574] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.448848] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:28 2020][8592512.455471] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.461744] ll_ost_io00_048 D ffff9c5914d530c0 0 68895 2 0x00000080 [Tue Mar 17 18:00:28 2020][8592512.469030] Call Trace: [Tue Mar 17 18:00:28 2020][8592512.471657] [] schedule+0x29/0x70 [Tue Mar 17 18:00:28 2020][8592512.476803] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.483855] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:28 2020][8592512.489872] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.497096] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:28 2020][8592512.502940] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.509654] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.516970] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:28 2020][8592512.523160] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.529972] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.537289] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.544865] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.552009] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:28 2020][8592512.558808] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:28 2020][8592512.565027] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.571762] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.578412] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.585486] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:28 2020][8592512.592805] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:28 2020][8592512.598573] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:28 2020][8592512.604093] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.611172] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.618920] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:28 2020][8592512.626175] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.634037] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.641009] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:28 2020][8592512.646445] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.652917] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.660490] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:28 2020][8592512.665566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.671833] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:28 2020][8592512.678448] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.684722] ll_ost_io01_054 D ffff9c5914d54100 0 68896 2 0x00000080 [Tue Mar 17 18:00:28 2020][8592512.692008] Call Trace: [Tue Mar 17 18:00:28 2020][8592512.694635] [] schedule+0x29/0x70 [Tue Mar 17 18:00:28 2020][8592512.699789] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.706839] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:28 2020][8592512.712857] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.720082] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:28 2020][8592512.725941] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.732656] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.739974] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:28 2020][8592512.746163] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:28 2020][8592512.752963] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.760287] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.767868] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.775010] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:28 2020][8592512.781812] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:28 2020][8592512.788030] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.794781] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.801433] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.808502] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:28 2020][8592512.815816] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:28 2020][8592512.821570] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:28 2020][8592512.827097] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.834186] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.841934] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:28 2020][8592512.849192] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.857055] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.864033] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:28 2020][8592512.869464] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.875940] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592512.883512] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:28 2020][8592512.888573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.894847] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:28 2020][8592512.901469] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592512.907746] ll_ost_io00_049 D ffff9c5914d55140 0 68897 2 0x00000080 [Tue Mar 17 18:00:28 2020][8592512.915019] Call Trace: [Tue Mar 17 18:00:28 2020][8592512.917649] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:28 2020][8592512.924270] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:28 2020][8592512.930647] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:28 2020][8592512.935961] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:28 2020][8592512.941456] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.948432] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592512.955835] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:28 2020][8592512.961853] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:28 2020][8592512.968217] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:28 2020][8592512.974833] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:28 2020][8592512.981974] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:28 2020][8592512.987221] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:28 2020][8592512.992892] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.000298] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:28 2020][8592513.007271] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.014595] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.021484] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.028803] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:28 2020][8592513.035687] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:28 2020][8592513.041918] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.048651] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.055315] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.062387] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:28 2020][8592513.069699] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:28 2020][8592513.075453] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:28 2020][8592513.082016] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.089445] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.096530] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.104283] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:28 2020][8592513.111537] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.119396] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.126377] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:28 2020][8592513.131807] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.138281] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.145854] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:28 2020][8592513.150914] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592513.157190] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:28 2020][8592513.163813] [] ? ret_from_fork+0x48/0x56 [Tue Mar 17 18:00:28 2020][8592513.169565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592513.175839] ll_ost_io00_050 D ffff9c5914d56180 0 68898 2 0x00000080 [Tue Mar 17 18:00:28 2020][8592513.183116] Call Trace: [Tue Mar 17 18:00:28 2020][8592513.185746] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:28 2020][8592513.191067] [] schedule+0x29/0x70 [Tue Mar 17 18:00:28 2020][8592513.196229] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:28 2020][8592513.202238] [] ? __find_get_block+0xbc/0x120 [Tue Mar 17 18:00:28 2020][8592513.208340] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:00:28 2020][8592513.213750] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:28 2020][8592513.219589] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:28 2020][8592513.224911] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:28 2020][8592513.231100] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:28 2020][8592513.236507] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:28 2020][8592513.241928] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:00:28 2020][8592513.247504] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:28 2020][8592513.252816] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:28 2020][8592513.258130] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:00:28 2020][8592513.264578] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592513.270684] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:00:28 2020][8592513.277564] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:28 2020][8592513.285143] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.293414] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:28 2020][8592513.298912] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.305899] [] osd_punch+0x28c/0x6b0 [osd_ldiskfs] [Tue Mar 17 18:00:28 2020][8592513.312531] [] ofd_object_punch+0x7df/0xd30 [ofd] [Tue Mar 17 18:00:28 2020][8592513.319069] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:28 2020][8592513.325389] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.332470] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.340230] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:28 2020][8592513.347488] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.355382] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.362346] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:28 2020][8592513.367775] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.374265] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:28 2020][8592513.381840] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:28 2020][8592513.386902] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592513.393175] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:28 2020][8592513.399796] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:28 2020][8592513.406072] ll_ost_io03_043 D ffff9c5913668000 0 68899 2 0x00000080 [Tue Mar 17 18:00:28 2020][8592513.413358] Call Trace: [Tue Mar 17 18:00:28 2020][8592513.415985] [] schedule+0x29/0x70 [Tue Mar 17 18:00:28 2020][8592513.421131] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.428180] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:29 2020][8592513.434197] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.441441] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.448154] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.455897] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:29 2020][8592513.462087] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.468883] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.476200] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.483783] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.490922] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:29 2020][8592513.497198] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:29 2020][8592513.504014] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:29 2020][8592513.510234] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.516966] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.523614] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.530683] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:29 2020][8592513.537997] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:29 2020][8592513.543752] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:29 2020][8592513.550312] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.557746] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.564833] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.572596] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:29 2020][8592513.579851] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.587711] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.594677] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:29 2020][8592513.600115] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.606588] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.614159] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:29 2020][8592513.619221] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592513.625497] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:29 2020][8592513.632117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592513.638407] ll_ost_io00_051 D ffff9c5913669040 0 68900 2 0x00000080 [Tue Mar 17 18:00:29 2020][8592513.645676] Call Trace: [Tue Mar 17 18:00:29 2020][8592513.648308] [] ? list_del+0xd/0x30 [Tue Mar 17 18:00:29 2020][8592513.653541] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:29 2020][8592513.658863] [] schedule+0x29/0x70 [Tue Mar 17 18:00:29 2020][8592513.664011] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:29 2020][8592513.670026] [] ? get_page_from_freelist+0x34c/0xa70 [Tue Mar 17 18:00:29 2020][8592513.676734] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:00:29 2020][8592513.682575] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:29 2020][8592513.687897] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:29 2020][8592513.694096] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:29 2020][8592513.699499] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:29 2020][8592513.704899] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:00:29 2020][8592513.710916] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:29 2020][8592513.716236] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:00:29 2020][8592513.723118] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592513.729218] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:00:29 2020][8592513.734806] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.741688] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.748487] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:29 2020][8592513.756081] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.764352] [] ldiskfs_quota_write+0xc6/0x1e0 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.771401] [] qtree_write_dquot+0xd5/0x180 [Tue Mar 17 18:00:29 2020][8592513.777416] [] v2_write_dquot+0x2b/0x30 [Tue Mar 17 18:00:29 2020][8592513.783083] [] dquot_commit+0xbd/0xd0 [Tue Mar 17 18:00:29 2020][8592513.788583] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.795551] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.802948] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:29 2020][8592513.809228] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.816110] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.824488] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.831720] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.839382] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:29 2020][8592513.844970] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.852639] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.860562] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.867970] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.875020] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:29 2020][8592513.880522] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.888712] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.895961] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592513.903275] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:29 2020][8592513.910072] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:29 2020][8592513.916289] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.923030] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.929683] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.936753] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:29 2020][8592513.944069] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:29 2020][8592513.949821] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:29 2020][8592513.955350] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.962453] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.970202] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:29 2020][8592513.977455] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.985317] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592513.992281] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:29 2020][8592513.997717] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.004190] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.011763] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:29 2020][8592514.016824] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.023101] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:29 2020][8592514.029735] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.036007] ll_ost_io03_044 D ffff9c591366a080 0 68901 2 0x00000080 [Tue Mar 17 18:00:29 2020][8592514.043280] Call Trace: [Tue Mar 17 18:00:29 2020][8592514.045910] [] schedule+0x29/0x70 [Tue Mar 17 18:00:29 2020][8592514.051061] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:29 2020][8592514.058113] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:29 2020][8592514.064129] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:29 2020][8592514.071357] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:29 2020][8592514.077201] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:29 2020][8592514.083913] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.091244] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:29 2020][8592514.097436] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:29 2020][8592514.104233] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.111551] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.119134] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.126281] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:29 2020][8592514.133075] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:29 2020][8592514.139293] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.146034] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.152686] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.159753] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:29 2020][8592514.165787] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.173220] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.180301] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.188049] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:29 2020][8592514.195302] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.203161] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.210129] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:29 2020][8592514.215565] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.222041] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.229626] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:29 2020][8592514.234687] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.240954] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:29 2020][8592514.247577] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.253852] ll_ost_io00_052 D ffff9c591366b0c0 0 68902 2 0x00000080 [Tue Mar 17 18:00:29 2020][8592514.261126] Call Trace: [Tue Mar 17 18:00:29 2020][8592514.263756] [] schedule+0x29/0x70 [Tue Mar 17 18:00:29 2020][8592514.268910] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.275967] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:29 2020][8592514.283110] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:29 2020][8592514.289142] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:29 2020][8592514.295334] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:29 2020][8592514.302130] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:29 2020][8592514.308345] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.315100] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.321748] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.328801] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:29 2020][8592514.334329] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.341417] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.349164] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:29 2020][8592514.356448] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.364309] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.371277] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:29 2020][8592514.376714] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.383188] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:29 2020][8592514.390758] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:29 2020][8592514.395821] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.402096] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:29 2020][8592514.408717] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:29 2020][8592514.414992] ll_ost_io03_045 D ffff9c591366c100 0 68903 2 0x00000080 [Tue Mar 17 18:00:29 2020][8592514.422284] Call Trace: [Tue Mar 17 18:00:29 2020][8592514.424924] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:30 2020][8592514.432585] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.437737] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:30 2020][8592514.444787] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:30 2020][8592514.450804] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:30 2020][8592514.458033] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:30 2020][8592514.464744] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592514.472489] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:30 2020][8592514.478677] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:30 2020][8592514.485490] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592514.492809] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:30 2020][8592514.500390] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592514.507530] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:30 2020][8592514.513803] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:30 2020][8592514.520600] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:30 2020][8592514.526816] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.533556] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.540204] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.547273] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:30 2020][8592514.554600] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:30 2020][8592514.560349] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:30 2020][8592514.565876] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.572965] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.580713] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:30 2020][8592514.587966] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.595827] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.602793] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:30 2020][8592514.608227] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.614702] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.622290] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592514.627353] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.633628] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592514.640246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.646513] ll_ost_create01 S ffff9c591366d140 0 68904 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592514.653790] Call Trace: [Tue Mar 17 18:00:30 2020][8592514.656418] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592514.662513] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.667691] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.674568] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592514.680368] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.686843] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.694415] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592514.699478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.705750] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592514.712372] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.718646] ll_ost_create00 S ffff9c591366e180 0 68905 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592514.725915] Call Trace: [Tue Mar 17 18:00:30 2020][8592514.728545] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592514.734647] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.739839] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.746718] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592514.752504] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.758949] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:30 2020][8592514.764643] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.772216] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592514.777277] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.783552] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592514.790174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.796449] ll_ost_create02 S ffff9c5905f38000 0 68906 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592514.803741] Call Trace: [Tue Mar 17 18:00:30 2020][8592514.806372] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592514.812486] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.817658] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.824538] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592514.830321] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.836796] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.844368] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592514.849429] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.855702] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592514.862339] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.868607] ll_ost_create03 S ffff9c5905f39040 0 68907 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592514.875893] Call Trace: [Tue Mar 17 18:00:30 2020][8592514.878521] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592514.884615] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.889792] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.896671] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592514.902454] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.908898] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:30 2020][8592514.914596] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.922183] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592514.927237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.933513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592514.940135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592514.946409] ldlm_cn02_004 S ffff9c5905f3a080 0 68908 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592514.953685] Call Trace: [Tue Mar 17 18:00:30 2020][8592514.956314] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592514.962407] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592514.967584] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.974466] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:30 2020][8592514.981100] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592514.986876] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592514.993351] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.000923] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592515.005984] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.012259] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592515.018880] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.025155] ldlm_cn00_004 S ffff9c5905f3b0c0 0 68909 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592515.032431] Call Trace: [Tue Mar 17 18:00:30 2020][8592515.035060] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592515.041152] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592515.046346] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.053228] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:30 2020][8592515.059849] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592515.065631] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.072106] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.079676] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592515.084739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.091016] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592515.097635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.103922] ldlm_cn01_004 S ffff9c5905f3c100 0 68910 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592515.111193] Call Trace: [Tue Mar 17 18:00:30 2020][8592515.113823] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:30 2020][8592515.119925] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592515.125104] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.131981] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:30 2020][8592515.137763] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.144239] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.151811] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592515.156871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.163160] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592515.169776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.176052] ldlm_cn00_005 D ffff9c5905f3d140 0 68911 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592515.183326] Call Trace: [Tue Mar 17 18:00:30 2020][8592515.185956] [] schedule+0x29/0x70 [Tue Mar 17 18:00:30 2020][8592515.191100] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:30 2020][8592515.197543] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.203640] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:30 2020][8592515.209391] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:30 2020][8592515.214883] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.221346] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.227799] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.235207] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.242262] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.249230] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.256384] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.263704] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:30 2020][8592515.270780] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.278287] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.285352] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:30 2020][8592515.291395] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.298719] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.305347] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:30 2020][8592515.311721] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:30 2020][8592515.318198] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.325525] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:30 2020][8592515.332869] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.339950] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.346943] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.354027] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.361879] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:30 2020][8592515.367311] [] ? ptlrpc_server_handle_req_in+0x8df/0xd60 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.375257] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.381733] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:30 2020][8592515.389305] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:30 2020][8592515.394366] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.400641] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:30 2020][8592515.407261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:30 2020][8592515.413537] ldlm_cn01_005 S ffff9c5905f3e180 0 68912 2 0x00000080 [Tue Mar 17 18:00:30 2020][8592515.420814] Call Trace: [Tue Mar 17 18:00:30 2020][8592515.423441] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.429535] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.434726] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.441607] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.447384] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.453857] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.461430] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.466491] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.472766] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.479387] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.485677] ll_ost_create01 S ffff9c58b5998000 0 68913 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.492945] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.495574] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.501668] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.506848] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.513725] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.519507] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.525984] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.533553] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.538614] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.544906] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.551519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.557793] ll_ost_create01 S ffff9c58b599b0c0 0 68940 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.565062] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.567691] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.573793] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.578970] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.585850] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.591633] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.598107] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.605692] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.610746] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.617018] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.623635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.629910] ll_ost_create01 S ffff9c58b599d140 0 68990 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.637187] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.639817] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.645919] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.651095] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.657972] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.663772] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.670248] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.677821] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.682882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.689157] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.695779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.702052] ll_ost_create02 S ffff9c58b599a080 0 69205 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.709328] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.711958] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.718059] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.723251] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.730131] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.735908] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.742380] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.749954] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.755019] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.761289] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.767910] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.774184] ldlm_bl_04 S ffff9c58b599e180 0 69224 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.781455] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.784085] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:31 2020][8592515.789940] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:31 2020][8592515.797358] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.802527] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.809580] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.815363] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.822936] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.827998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.834272] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.840893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.847182] ldlm_bl_05 S ffff9c58b599c100 0 69232 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.854453] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.857083] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:31 2020][8592515.862939] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:31 2020][8592515.870344] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.875520] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.882573] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.888354] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.895926] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.900987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.907264] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.913886] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.920158] ldlm_cn02_005 S ffff9c8a1f3c9040 0 69306 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.927435] Call Trace: [Tue Mar 17 18:00:31 2020][8592515.930065] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592515.936166] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592515.941344] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.948222] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592515.954006] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.960487] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592515.968077] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592515.973141] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.979413] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592515.986034] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592515.992311] ldlm_cn03_005 S ffff9c8a1f3c8000 0 69307 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592515.999585] Call Trace: [Tue Mar 17 18:00:31 2020][8592516.002214] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592516.008318] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592516.013493] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.020374] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592516.026170] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.032646] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.040219] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592516.045282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.051555] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592516.058177] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.064453] ll_ost_create00 S ffff9c8a1f3cd140 0 69325 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592516.071729] Call Trace: [Tue Mar 17 18:00:31 2020][8592516.074356] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592516.080449] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592516.085640] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.092513] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592516.098288] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.104735] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:31 2020][8592516.110429] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.118002] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592516.123065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.129339] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592516.135960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.142235] ll_ost_create00 S ffff9c8a1f3ca080 0 69326 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592516.149530] Call Trace: [Tue Mar 17 18:00:31 2020][8592516.152159] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:31 2020][8592516.158261] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592516.163437] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.170316] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:31 2020][8592516.176100] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.182546] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:31 2020][8592516.188240] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.195813] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:31 2020][8592516.200875] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.207163] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:31 2020][8592516.213779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.220052] ldlm_cn02_006 D ffff9c7a34459040 0 69427 2 0x00000080 [Tue Mar 17 18:00:31 2020][8592516.227321] Call Trace: [Tue Mar 17 18:00:31 2020][8592516.229951] [] schedule+0x29/0x70 [Tue Mar 17 18:00:31 2020][8592516.235100] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:31 2020][8592516.241546] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:31 2020][8592516.247648] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:31 2020][8592516.253404] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:31 2020][8592516.258905] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.265375] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.271827] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.279236] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.286292] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.293268] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.300419] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.307743] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:31 2020][8592516.314834] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.322338] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.329403] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:31 2020][8592516.335447] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.342775] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.349399] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:31 2020][8592516.355763] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:31 2020][8592516.362235] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.369576] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:31 2020][8592516.376913] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.383995] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.390987] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.398071] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.405946] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.412916] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:31 2020][8592516.418353] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:31 2020][8592516.424827] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.432398] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.437460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.443734] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.450356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.456627] ll_ost_create02 S ffff9c7a3445d140 0 69477 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.463896] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.466527] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.472627] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.477820] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.484700] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.490477] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.496949] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.504522] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.509583] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.515857] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.522479] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.528766] ll_ost_create03 S ffff9c7a3445a080 0 69524 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.536039] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.538669] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.544770] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.549947] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.556826] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.562609] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.569084] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.576656] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.581720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.588005] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.594622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.600895] ll_ost_create03 S ffff9c5997fc1040 0 69530 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.608162] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.610792] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.616887] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.622062] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.628941] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.634727] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.641199] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.648787] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.653849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.660124] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.666745] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.673020] ldlm_cn01_006 S ffff9c5997fc6180 0 69531 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.680297] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.682928] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.689028] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.694205] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.701086] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.706880] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.713350] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.720921] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.725984] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.732257] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.738877] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.745145] ldlm_cn01_007 S ffff9c5a36f7e180 0 69656 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.752419] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.755050] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.761144] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.766334] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.773216] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.779000] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.785475] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.793047] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.798107] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.804382] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.811003] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.817278] ldlm_cn00_006 S ffff9c5997fc30c0 0 69929 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.824553] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.827184] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.833287] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.838477] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.845357] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.851141] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.857615] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.865187] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.870248] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.876524] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.883145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.889433] ldlm_bl_06 S ffff9c79cab930c0 0 70021 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.896705] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.899335] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:32 2020][8592516.905192] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:32 2020][8592516.912595] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.917771] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.924825] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.930606] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.938180] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592516.943240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.949530] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592516.956144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592516.962420] ll_ost_create00 S ffff9c5a147d0000 0 70252 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592516.969696] Call Trace: [Tue Mar 17 18:00:32 2020][8592516.972324] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592516.978417] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592516.983586] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592516.990465] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592516.996248] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.002695] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:32 2020][8592517.008405] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.015980] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.021043] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.027317] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.033937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.040211] ldlm_cn03_006 S ffff9c8a24f45140 0 70323 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.047479] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.050110] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592517.056211] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.061388] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.068281] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:32 2020][8592517.074897] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592517.080679] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.087153] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.094726] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.099788] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.106062] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.112682] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.118957] ldlm_bl_07 S ffff9c4b695cb0c0 0 70500 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.126227] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.128855] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:32 2020][8592517.134712] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:32 2020][8592517.142129] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.147300] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.154352] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592517.160133] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.167710] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.172770] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.179043] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.185667] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.191954] ldlm_bl_08 S ffff9c8a31a72080 0 70798 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.199224] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.201854] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:32 2020][8592517.207703] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:32 2020][8592517.215098] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.220274] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.227329] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592517.233109] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.240681] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.245744] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.252036] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.258656] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.264931] ldlm_bl_09 S ffff9c4b69b1c100 0 70868 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.272200] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.274828] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:32 2020][8592517.280677] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:32 2020][8592517.288081] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.293255] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.300311] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592517.306093] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.313696] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.318752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.325042] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.331656] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.337923] ll_ost_create02 S ffff9c6a2cedc100 0 71564 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.345200] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.347829] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592517.353931] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.359127] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.366004] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:32 2020][8592517.371806] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.378294] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:32 2020][8592517.385869] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:32 2020][8592517.390930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.397205] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:32 2020][8592517.403826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:32 2020][8592517.410099] ldlm_cb01_002 S ffff9c6a37359040 0 71784 2 0x00000080 [Tue Mar 17 18:00:32 2020][8592517.417375] Call Trace: [Tue Mar 17 18:00:32 2020][8592517.420005] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:32 2020][8592517.426107] [] schedule+0x29/0x70 [Tue Mar 17 18:00:32 2020][8592517.431300] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.438178] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:33 2020][8592517.443955] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.450400] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:33 2020][8592517.456096] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.463670] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592517.468730] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.475004] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592517.481625] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.487902] ldlm_cb01_003 S ffff9c6a3735a080 0 71785 2 0x00000080 [Tue Mar 17 18:00:33 2020][8592517.495194] Call Trace: [Tue Mar 17 18:00:33 2020][8592517.497823] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:33 2020][8592517.503917] [] schedule+0x29/0x70 [Tue Mar 17 18:00:33 2020][8592517.509094] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.515973] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:33 2020][8592517.521755] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.528201] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:33 2020][8592517.533897] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.541470] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592517.546532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.552807] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592517.559426] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.565702] ll_ost_io01_055 D ffff9c5a39d430c0 0 71821 2 0x00000080 [Tue Mar 17 18:00:33 2020][8592517.572977] Call Trace: [Tue Mar 17 18:00:33 2020][8592517.575608] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:33 2020][8592517.582229] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:33 2020][8592517.588590] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:33 2020][8592517.593912] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:33 2020][8592517.599413] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.606391] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.613811] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:33 2020][8592517.619834] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:33 2020][8592517.626200] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:33 2020][8592517.632818] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:33 2020][8592517.638310] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:33 2020][8592517.643539] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:33 2020][8592517.650679] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:33 2020][8592517.655914] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:33 2020][8592517.661588] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.668997] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592517.675979] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.683297] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.690179] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.697493] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:33 2020][8592517.704374] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:33 2020][8592517.710592] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.717330] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.723977] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.731046] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:33 2020][8592517.738363] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:33 2020][8592517.744130] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:33 2020][8592517.749658] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.756739] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.764487] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:33 2020][8592517.771738] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.779599] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.786567] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:33 2020][8592517.792002] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.798475] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.806048] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592517.811125] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.817402] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592517.824024] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592517.830298] ll_ost_io01_056 D ffff9c5a39d42080 0 71822 2 0x00000080 [Tue Mar 17 18:00:33 2020][8592517.837572] Call Trace: [Tue Mar 17 18:00:33 2020][8592517.840204] [] schedule+0x29/0x70 [Tue Mar 17 18:00:33 2020][8592517.845356] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:33 2020][8592517.852405] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:33 2020][8592517.858416] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592517.865640] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:33 2020][8592517.871499] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:33 2020][8592517.878216] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.885531] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:33 2020][8592517.891720] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592517.898519] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.905834] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.913416] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592517.920558] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:33 2020][8592517.927351] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:33 2020][8592517.933569] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.940294] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:33 2020][8592517.946251] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.952902] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.959953] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:33 2020][8592517.965480] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.972566] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.980315] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:33 2020][8592517.987565] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592517.995428] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.002392] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:33 2020][8592518.007845] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.014321] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.021894] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592518.026955] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592518.033228] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592518.039852] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592518.046125] ll_ost_io01_057 D ffff9c5917d8b0c0 0 71823 2 0x00000080 [Tue Mar 17 18:00:33 2020][8592518.053401] Call Trace: [Tue Mar 17 18:00:33 2020][8592518.056031] [] schedule+0x29/0x70 [Tue Mar 17 18:00:33 2020][8592518.061182] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.068250] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:33 2020][8592518.074267] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.081494] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:33 2020][8592518.087337] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.094052] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.101366] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:33 2020][8592518.107556] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.114356] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.121680] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.129263] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.136425] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:33 2020][8592518.143223] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:33 2020][8592518.149439] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.156179] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.162827] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.169879] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:33 2020][8592518.175405] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.182492] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.190239] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:33 2020][8592518.197492] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.205368] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.212334] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:33 2020][8592518.217764] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.224239] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.231810] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592518.236871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592518.243146] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592518.249768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592518.256041] ll_ost_io01_058 D ffff9c5917d8c100 0 71824 2 0x00000080 [Tue Mar 17 18:00:33 2020][8592518.263325] Call Trace: [Tue Mar 17 18:00:33 2020][8592518.265958] [] schedule+0x29/0x70 [Tue Mar 17 18:00:33 2020][8592518.271108] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.278176] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:33 2020][8592518.284194] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.291422] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.298133] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.305876] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:33 2020][8592518.312065] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:33 2020][8592518.318878] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.326196] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.333793] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:33 2020][8592518.340950] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:33 2020][8592518.347229] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:33 2020][8592518.353763] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:33 2020][8592518.360086] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.367172] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.374918] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:33 2020][8592518.382172] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.390034] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.397005] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:33 2020][8592518.402457] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.408934] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:33 2020][8592518.416507] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:33 2020][8592518.421568] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:33 2020][8592518.427841] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:33 2020][8592518.434464] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592518.440737] ll_ost_io01_059 D ffff9c5917d8e180 0 71825 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592518.448013] Call Trace: [Tue Mar 17 18:00:34 2020][8592518.450643] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592518.455797] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.462863] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:34 2020][8592518.468873] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.476100] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:34 2020][8592518.481942] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.488655] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.495972] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:34 2020][8592518.502161] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.508959] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.516276] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.523859] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.531029] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:34 2020][8592518.537828] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:34 2020][8592518.544046] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.550751] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:34 2020][8592518.556273] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.563361] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.571107] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:34 2020][8592518.578359] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.586220] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.593185] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:34 2020][8592518.598638] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.605112] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.612687] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592518.617748] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592518.624026] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592518.630643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592518.636917] ll_ost_io01_060 D ffff9c5917d8d140 0 71826 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592518.644186] Call Trace: [Tue Mar 17 18:00:34 2020][8592518.646814] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592518.651965] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.659038] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:34 2020][8592518.665053] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.672278] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:34 2020][8592518.678122] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.684836] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.692153] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:34 2020][8592518.698343] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.705140] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.712457] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.720039] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.727202] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:34 2020][8592518.733997] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:34 2020][8592518.740216] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.746955] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.753604] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.760653] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:34 2020][8592518.766694] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.774127] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.781208] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.788955] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:34 2020][8592518.796223] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.804084] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.811050] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:34 2020][8592518.816486] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.822962] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.830534] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592518.835595] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592518.841870] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592518.848489] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592518.854764] ll_ost_io01_061 D ffff9c5997fc2080 0 71827 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592518.862033] Call Trace: [Tue Mar 17 18:00:34 2020][8592518.864661] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592518.869805] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.876857] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:34 2020][8592518.882875] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.890104] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.896813] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.904557] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:34 2020][8592518.910746] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592518.917546] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.924884] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.932459] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592518.939601] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:34 2020][8592518.945875] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:34 2020][8592518.952407] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:34 2020][8592518.958714] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.965803] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.973556] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:34 2020][8592518.980809] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.988669] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592518.995650] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:34 2020][8592519.001078] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.007553] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.015127] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592519.020188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.026461] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592519.033082] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.039357] ldlm_cn02_007 S ffff9c6a2ced9040 0 72066 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592519.046627] Call Trace: [Tue Mar 17 18:00:34 2020][8592519.049255] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:34 2020][8592519.055357] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592519.060548] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.067429] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:34 2020][8592519.073213] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.079688] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.087259] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592519.092319] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.098596] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592519.105215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.111490] ldlm_cn03_007 S ffff9c8a1f7d8000 0 72311 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592519.118777] Call Trace: [Tue Mar 17 18:00:34 2020][8592519.121405] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:34 2020][8592519.127506] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592519.132685] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.139562] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:34 2020][8592519.145347] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.151821] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.159393] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592519.164454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.170728] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592519.177365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.183633] ldlm_cn00_007 S ffff9c8a1f7db0c0 0 72312 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592519.190908] Call Trace: [Tue Mar 17 18:00:34 2020][8592519.193539] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:34 2020][8592519.199641] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592519.204817] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.211697] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:34 2020][8592519.217479] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.223954] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.231526] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592519.236599] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.242870] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592519.249493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.255765] ldlm_cn02_008 S ffff9c6a3735d140 0 72313 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592519.263033] Call Trace: [Tue Mar 17 18:00:34 2020][8592519.265663] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:34 2020][8592519.271766] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592519.276942] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.283821] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:34 2020][8592519.289604] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.296094] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:34 2020][8592519.303667] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:34 2020][8592519.308729] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.315004] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:34 2020][8592519.321639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:34 2020][8592519.327907] ll_ost_io01_062 D ffff9c79cb1a6180 0 73693 2 0x00000080 [Tue Mar 17 18:00:34 2020][8592519.335197] Call Trace: [Tue Mar 17 18:00:34 2020][8592519.337829] [] schedule+0x29/0x70 [Tue Mar 17 18:00:34 2020][8592519.342976] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:34 2020][8592519.350029] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:34 2020][8592519.356075] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592519.363303] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:34 2020][8592519.369147] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:34 2020][8592519.375879] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592519.383193] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:34 2020][8592519.389384] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:34 2020][8592519.396181] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592519.403499] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:34 2020][8592519.411079] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:34 2020][8592519.418220] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:34 2020][8592519.425031] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:34 2020][8592519.431248] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.437989] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.444635] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.451705] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:35 2020][8592519.459019] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:35 2020][8592519.464772] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:35 2020][8592519.470290] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.477372] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.485127] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:35 2020][8592519.492395] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.500259] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.507222] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:35 2020][8592519.512660] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.519134] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.526705] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:35 2020][8592519.531768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.538041] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:35 2020][8592519.544663] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.550938] ll_ost_io01_063 D ffff9c79cb1a4100 0 73694 2 0x00000080 [Tue Mar 17 18:00:35 2020][8592519.558230] Call Trace: [Tue Mar 17 18:00:35 2020][8592519.560862] [] schedule+0x29/0x70 [Tue Mar 17 18:00:35 2020][8592519.566013] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.573063] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:35 2020][8592519.579081] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.586309] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.593022] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.600769] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:35 2020][8592519.606952] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.613750] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.621081] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.628658] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.635805] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:35 2020][8592519.642082] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:35 2020][8592519.648875] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:35 2020][8592519.655091] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.661824] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.668472] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.675522] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:35 2020][8592519.681050] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.688150] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.695903] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:35 2020][8592519.703153] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.711015] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.717979] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:35 2020][8592519.723417] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.729891] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.737464] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:35 2020][8592519.742523] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.748798] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:35 2020][8592519.755436] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.761703] ll_ost_io01_064 D ffff9c79cb1a2080 0 73695 2 0x00000080 [Tue Mar 17 18:00:35 2020][8592519.768980] Call Trace: [Tue Mar 17 18:00:35 2020][8592519.771608] [] schedule+0x29/0x70 [Tue Mar 17 18:00:35 2020][8592519.776753] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.783804] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:35 2020][8592519.789822] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.797046] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:35 2020][8592519.802892] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.809603] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.816935] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:35 2020][8592519.823119] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.829915] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.837232] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.844815] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592519.851962] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:35 2020][8592519.858758] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:35 2020][8592519.864973] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.871705] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.878354] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.885417] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:35 2020][8592519.890938] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.898019] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.905772] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:35 2020][8592519.913028] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.920886] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.927853] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:35 2020][8592519.933292] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.939763] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592519.947337] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:35 2020][8592519.952413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.958682] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:35 2020][8592519.965301] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592519.971578] ll_ost_io01_065 D ffff9c5a2862e180 0 73696 2 0x00000080 [Tue Mar 17 18:00:35 2020][8592519.978852] Call Trace: [Tue Mar 17 18:00:35 2020][8592519.981484] [] schedule+0x29/0x70 [Tue Mar 17 18:00:35 2020][8592519.986635] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:35 2020][8592519.993686] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:35 2020][8592519.999704] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.006929] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:35 2020][8592520.012787] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.019502] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.026820] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:35 2020][8592520.033009] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.039808] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.047130] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.054705] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.061846] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:35 2020][8592520.068640] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:35 2020][8592520.074856] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.081583] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:35 2020][8592520.087539] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.094190] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.101241] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:35 2020][8592520.106767] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.113854] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.121602] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:35 2020][8592520.128855] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.136715] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.143681] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:35 2020][8592520.149132] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.155609] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.163182] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:35 2020][8592520.168243] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592520.174517] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:35 2020][8592520.181137] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592520.187413] ll_ost_io01_066 D ffff9c69b5896180 0 73697 2 0x00000080 [Tue Mar 17 18:00:35 2020][8592520.194690] Call Trace: [Tue Mar 17 18:00:35 2020][8592520.197318] [] schedule+0x29/0x70 [Tue Mar 17 18:00:35 2020][8592520.202462] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.209530] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:35 2020][8592520.215549] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.222772] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:35 2020][8592520.228618] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.235332] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.242647] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:35 2020][8592520.248836] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:35 2020][8592520.255633] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.262950] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.270534] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:35 2020][8592520.277683] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:35 2020][8592520.284475] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:35 2020][8592520.290693] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.297433] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.304080] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.311149] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:35 2020][8592520.318462] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:35 2020][8592520.324234] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:35 2020][8592520.330795] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.338245] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.345342] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.353090] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:35 2020][8592520.360342] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.368205] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.375170] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:35 2020][8592520.380953] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.387427] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:35 2020][8592520.394998] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:35 2020][8592520.400059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:35 2020][8592520.406335] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:35 2020][8592520.412971] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592520.419239] ll_ost_io01_067 D ffff9c69b5890000 0 73698 2 0x00000080 [Tue Mar 17 18:00:36 2020][8592520.426524] Call Trace: [Tue Mar 17 18:00:36 2020][8592520.429153] [] schedule+0x29/0x70 [Tue Mar 17 18:00:36 2020][8592520.434304] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:36 2020][8592520.441357] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:36 2020][8592520.447373] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:36 2020][8592520.454603] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:36 2020][8592520.461314] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.469056] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:36 2020][8592520.475260] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:36 2020][8592520.482052] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.489369] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.496951] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.504091] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:36 2020][8592520.510366] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:36 2020][8592520.517160] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:36 2020][8592520.523376] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.530106] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.536756] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.543838] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:36 2020][8592520.551148] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:36 2020][8592520.556902] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:36 2020][8592520.563461] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.570895] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.577976] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.585723] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:36 2020][8592520.592976] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.600839] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.607802] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:36 2020][8592520.613253] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.619728] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.627302] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:36 2020][8592520.632362] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592520.638638] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:36 2020][8592520.645259] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592520.651533] ll_ost_io01_068 D ffff9c69b5894100 0 73699 2 0x00000080 [Tue Mar 17 18:00:36 2020][8592520.658799] Call Trace: [Tue Mar 17 18:00:36 2020][8592520.661430] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:36 2020][8592520.668045] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:36 2020][8592520.674420] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:36 2020][8592520.679736] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:36 2020][8592520.685236] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.692215] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.699616] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:36 2020][8592520.705637] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:36 2020][8592520.711998] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:36 2020][8592520.718618] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.725322] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:36 2020][8592520.732464] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:36 2020][8592520.737713] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:36 2020][8592520.743381] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.750780] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:36 2020][8592520.757749] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.765065] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.771947] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.779269] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:36 2020][8592520.786150] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:36 2020][8592520.792367] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.799104] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.805770] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.812821] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:36 2020][8592520.818347] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.825428] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.833174] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:36 2020][8592520.840430] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.848288] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.855255] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:36 2020][8592520.860692] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.867165] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592520.874751] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:36 2020][8592520.879806] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592520.886082] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:36 2020][8592520.892702] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592520.898977] ll_ost_io01_069 D ffff9c69b5891040 0 73700 2 0x00000080 [Tue Mar 17 18:00:36 2020][8592520.906252] Call Trace: [Tue Mar 17 18:00:36 2020][8592520.908882] [] schedule+0x29/0x70 [Tue Mar 17 18:00:36 2020][8592520.914036] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:00:36 2020][8592520.921521] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:36 2020][8592520.927537] [] raid5_make_request+0x1a5/0xca0 [raid456] [Tue Mar 17 18:00:36 2020][8592520.934603] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:00:36 2020][8592520.939745] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:36 2020][8592520.945768] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592520.953606] [] md_handle_request+0xd0/0x150 [Tue Mar 17 18:00:36 2020][8592520.959620] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 18:00:36 2020][8592520.966760] [] md_make_request+0x79/0x190 [Tue Mar 17 18:00:36 2020][8592520.972601] [] generic_make_request+0x147/0x380 [Tue Mar 17 18:00:36 2020][8592520.978964] [] ? md_mergeable_bvec+0x46/0x50 [Tue Mar 17 18:00:36 2020][8592520.985065] [] submit_bio+0x70/0x150 [Tue Mar 17 18:00:36 2020][8592520.990489] [] ? lprocfs_oh_tally+0x17/0x40 [obdclass] [Tue Mar 17 18:00:36 2020][8592520.997481] [] osd_submit_bio+0x1c/0x60 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.004364] [] osd_do_bio.isra.35+0x9b5/0xab0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.011766] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:36 2020][8592521.017959] [] ? jbd2_journal_add_journal_head+0x65/0x150 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.025790] [] ? recalc_bh_state.part.21+0x70/0x90 [Tue Mar 17 18:00:36 2020][8592521.032413] [] ? jbd2_journal_get_write_access+0x32/0x40 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.040165] [] osd_write_commit+0x3ec/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.047402] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.054737] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:36 2020][8592521.061533] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:36 2020][8592521.067751] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.074506] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.081155] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.088207] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:36 2020][8592521.093734] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.100823] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.108570] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:36 2020][8592521.115821] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.123683] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.130647] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:36 2020][8592521.136101] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.142574] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.150149] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:36 2020][8592521.155210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592521.161485] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:36 2020][8592521.168105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592521.174379] ll_ost_io01_070 D ffff9c69b5892080 0 73701 2 0x00000080 [Tue Mar 17 18:00:36 2020][8592521.181647] Call Trace: [Tue Mar 17 18:00:36 2020][8592521.184275] [] schedule+0x29/0x70 [Tue Mar 17 18:00:36 2020][8592521.189420] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.196487] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:36 2020][8592521.202498] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.209724] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:36 2020][8592521.215566] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.222280] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.229596] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:36 2020][8592521.235787] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:36 2020][8592521.242585] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.249908] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.257491] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:36 2020][8592521.264644] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:36 2020][8592521.271441] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:36 2020][8592521.277659] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.284400] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.291050] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.298122] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:36 2020][8592521.305431] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:36 2020][8592521.311184] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:36 2020][8592521.316712] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.323815] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.331579] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:36 2020][8592521.338848] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.346713] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.353678] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:36 2020][8592521.359118] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.365613] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:36 2020][8592521.373184] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:36 2020][8592521.378238] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592521.384513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:36 2020][8592521.391135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:36 2020][8592521.397424] ll_ost_io01_071 D ffff9c69b58930c0 0 73702 2 0x00000080 [Tue Mar 17 18:00:36 2020][8592521.404693] Call Trace: [Tue Mar 17 18:00:36 2020][8592521.407325] [] schedule+0x29/0x70 [Tue Mar 17 18:00:36 2020][8592521.412477] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.419527] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:37 2020][8592521.425546] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.432769] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:37 2020][8592521.438614] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.445328] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.452642] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:37 2020][8592521.458847] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.465650] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.472974] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.480556] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.487702] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:37 2020][8592521.494499] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:37 2020][8592521.500717] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.507427] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:37 2020][8592521.512952] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.520044] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.527809] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:37 2020][8592521.535066] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.542928] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.549892] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:37 2020][8592521.555330] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.561806] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.569376] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:37 2020][8592521.574436] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592521.580711] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:37 2020][8592521.587332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592521.593621] ll_ost_io01_072 D ffff9c5a39ea5140 0 73703 2 0x00000080 [Tue Mar 17 18:00:37 2020][8592521.600890] Call Trace: [Tue Mar 17 18:00:37 2020][8592521.603529] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.611192] [] schedule+0x29/0x70 [Tue Mar 17 18:00:37 2020][8592521.616342] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.623396] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:37 2020][8592521.629410] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.636641] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.643352] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.651104] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:37 2020][8592521.657308] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.664108] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.671423] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.678998] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.686146] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:37 2020][8592521.692420] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:37 2020][8592521.699214] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:37 2020][8592521.705432] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.712173] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.718821] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.725903] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:37 2020][8592521.733220] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:37 2020][8592521.738975] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:37 2020][8592521.744503] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.751588] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.759337] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:37 2020][8592521.766591] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.774450] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.781418] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:37 2020][8592521.786852] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.793340] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.800914] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:37 2020][8592521.805969] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592521.812242] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:37 2020][8592521.818864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592521.825139] ll_ost_io01_073 D ffff9c5997fc5140 0 73704 2 0x00000080 [Tue Mar 17 18:00:37 2020][8592521.832423] Call Trace: [Tue Mar 17 18:00:37 2020][8592521.835053] [] schedule+0x29/0x70 [Tue Mar 17 18:00:37 2020][8592521.840204] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.847256] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:37 2020][8592521.853289] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.860519] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.867229] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.874974] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:37 2020][8592521.881162] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592521.887960] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.895276] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.902850] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592521.909991] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:37 2020][8592521.916264] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:37 2020][8592521.923075] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:37 2020][8592521.929293] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.936034] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.942688] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.949758] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:37 2020][8592521.957074] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:37 2020][8592521.962828] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:37 2020][8592521.969389] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.976822] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.983909] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592521.991671] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:37 2020][8592521.998925] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.006786] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.013752] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:37 2020][8592522.019189] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.025662] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.033236] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:37 2020][8592522.038297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592522.044571] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:37 2020][8592522.051193] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592522.057482] ll_ost_io01_074 D ffff9c5997fc4100 0 73705 2 0x00000080 [Tue Mar 17 18:00:37 2020][8592522.064750] Call Trace: [Tue Mar 17 18:00:37 2020][8592522.067383] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:37 2020][8592522.074003] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:37 2020][8592522.080365] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:37 2020][8592522.085688] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:37 2020][8592522.091189] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.098165] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.105568] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:37 2020][8592522.111851] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.118754] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.127112] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.134349] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.142013] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:37 2020][8592522.147600] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.155270] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.163200] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.170610] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.177674] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.185861] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:37 2020][8592522.192846] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.200074] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.207388] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:37 2020][8592522.214183] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:37 2020][8592522.220401] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.227112] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:37 2020][8592522.232865] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:37 2020][8592522.238393] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.245481] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.253226] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:37 2020][8592522.260494] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.268357] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.275323] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:37 2020][8592522.280760] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.287234] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:37 2020][8592522.294806] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:37 2020][8592522.299869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592522.306143] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:37 2020][8592522.312764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:37 2020][8592522.319038] ll_ost_io01_075 D ffff9c5a39d52080 0 73706 2 0x00000080 [Tue Mar 17 18:00:37 2020][8592522.326350] Call Trace: [Tue Mar 17 18:00:37 2020][8592522.328977] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:37 2020][8592522.335593] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:37 2020][8592522.341953] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:37 2020][8592522.347274] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:37 2020][8592522.352777] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.359773] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:37 2020][8592522.367173] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:37 2020][8592522.373193] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:37 2020][8592522.379555] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:37 2020][8592522.386183] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:37 2020][8592522.391674] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:37 2020][8592522.396911] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:37 2020][8592522.404051] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:37 2020][8592522.409286] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:37 2020][8592522.414958] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.422356] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.429329] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.436652] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.443541] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.450871] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:38 2020][8592522.457754] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:38 2020][8592522.463971] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.470709] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.477359] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.484435] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:38 2020][8592522.491749] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:38 2020][8592522.497507] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:00:38 2020][8592522.503171] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:38 2020][8592522.508700] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.515801] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.523552] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:38 2020][8592522.530806] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.538664] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.545631] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:38 2020][8592522.551414] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.557888] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.565459] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:38 2020][8592522.570520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.576796] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:38 2020][8592522.583431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.589701] ll_ost_io01_076 D ffff9c5a39d51040 0 73707 2 0x00000080 [Tue Mar 17 18:00:38 2020][8592522.596974] Call Trace: [Tue Mar 17 18:00:38 2020][8592522.599607] [] schedule+0x29/0x70 [Tue Mar 17 18:00:38 2020][8592522.604760] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.611815] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.618960] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:38 2020][8592522.624978] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:38 2020][8592522.631166] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:38 2020][8592522.637961] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:38 2020][8592522.644193] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.650925] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.657574] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.664623] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:38 2020][8592522.670665] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.678097] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.685179] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.692935] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:38 2020][8592522.700187] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.708048] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.715025] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:38 2020][8592522.720456] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.726932] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.734504] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:38 2020][8592522.739566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.745840] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:38 2020][8592522.752460] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.758726] ll_ost_io01_077 D ffff9c5a39d530c0 0 73708 2 0x00000080 [Tue Mar 17 18:00:38 2020][8592522.765995] Call Trace: [Tue Mar 17 18:00:38 2020][8592522.768623] [] schedule+0x29/0x70 [Tue Mar 17 18:00:38 2020][8592522.773768] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.780835] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:38 2020][8592522.786845] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.794072] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:38 2020][8592522.799915] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.806629] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.813945] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:38 2020][8592522.820133] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.826931] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.834247] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.841845] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.848986] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:38 2020][8592522.855782] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:38 2020][8592522.861999] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.868710] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:38 2020][8592522.874235] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.881324] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.889078] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:38 2020][8592522.896333] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.904192] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.911174] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:38 2020][8592522.916611] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.923084] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592522.930658] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:38 2020][8592522.935720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.941995] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:38 2020][8592522.948615] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592522.954891] ll_ost_io01_078 D ffff9c5a368fc100 0 73709 2 0x00000080 [Tue Mar 17 18:00:38 2020][8592522.962165] Call Trace: [Tue Mar 17 18:00:38 2020][8592522.964803] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592522.972465] [] schedule+0x29/0x70 [Tue Mar 17 18:00:38 2020][8592522.977632] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.984685] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:38 2020][8592522.990696] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592522.997922] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:38 2020][8592523.004633] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.012376] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:38 2020][8592523.018566] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592523.025364] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.032681] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.040279] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.047420] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:38 2020][8592523.053695] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:38 2020][8592523.060488] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:38 2020][8592523.066708] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.073446] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.080095] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.087145] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:38 2020][8592523.092672] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.099759] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.107522] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:38 2020][8592523.114775] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.122635] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.129604] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:38 2020][8592523.135040] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.141509] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.149085] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:38 2020][8592523.154147] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592523.160422] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:38 2020][8592523.167043] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592523.173333] ll_ost_io01_079 D ffff9c5a368f8000 0 73710 2 0x00000080 [Tue Mar 17 18:00:38 2020][8592523.180602] Call Trace: [Tue Mar 17 18:00:38 2020][8592523.183233] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:38 2020][8592523.189854] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:38 2020][8592523.196215] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:38 2020][8592523.201536] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:38 2020][8592523.207042] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.214015] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.221419] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:38 2020][8592523.227437] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:38 2020][8592523.233812] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:38 2020][8592523.240433] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:38 2020][8592523.245926] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:38 2020][8592523.251163] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:38 2020][8592523.258303] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:38 2020][8592523.263539] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:38 2020][8592523.269212] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.276611] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:38 2020][8592523.283580] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.290895] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.297794] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:38 2020][8592523.305107] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:38 2020][8592523.311989] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:38 2020][8592523.318207] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.324917] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:38 2020][8592523.330459] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.337539] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.345287] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:38 2020][8592523.352539] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.360424] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.367406] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:38 2020][8592523.372837] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.379310] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:38 2020][8592523.386882] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:38 2020][8592523.391945] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592523.398220] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:38 2020][8592523.404840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:38 2020][8592523.411114] ll_ost_io01_080 D ffff9c5a368fb0c0 0 73711 2 0x00000080 [Tue Mar 17 18:00:38 2020][8592523.418381] Call Trace: [Tue Mar 17 18:00:39 2020][8592523.421012] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:39 2020][8592523.427634] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:39 2020][8592523.434010] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:39 2020][8592523.439324] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:39 2020][8592523.444819] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.451795] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.459199] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:39 2020][8592523.465218] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:39 2020][8592523.471578] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:39 2020][8592523.478196] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:39 2020][8592523.483690] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:39 2020][8592523.488940] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:39 2020][8592523.496076] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:39 2020][8592523.501310] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:39 2020][8592523.506983] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.514381] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.521352] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.528670] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.535559] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.542880] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:39 2020][8592523.549760] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:39 2020][8592523.555993] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.562724] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.569374] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.576424] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:39 2020][8592523.581951] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.589041] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.596792] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:39 2020][8592523.604051] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.611916] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.618883] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:39 2020][8592523.624319] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.630808] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.638382] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:39 2020][8592523.643444] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592523.649719] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:39 2020][8592523.656340] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592523.662616] ll_ost_io01_081 D ffff9c5a368fe180 0 73712 2 0x00000080 [Tue Mar 17 18:00:39 2020][8592523.669900] Call Trace: [Tue Mar 17 18:00:39 2020][8592523.672528] [] schedule+0x29/0x70 [Tue Mar 17 18:00:39 2020][8592523.677673] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.684740] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:39 2020][8592523.690758] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.697987] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.704700] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.712447] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:39 2020][8592523.718640] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.725437] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.732752] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.740337] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.747484] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:39 2020][8592523.753771] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:39 2020][8592523.760309] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:39 2020][8592523.766613] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.773701] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.781448] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:39 2020][8592523.788700] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.796561] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.803529] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:39 2020][8592523.808963] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.815438] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.823026] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:39 2020][8592523.828087] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592523.834354] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:39 2020][8592523.840976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592523.847252] ll_ost_io01_082 D ffff9c5a368fa080 0 73713 2 0x00000080 [Tue Mar 17 18:00:39 2020][8592523.854528] Call Trace: [Tue Mar 17 18:00:39 2020][8592523.857156] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:39 2020][8592523.863779] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:39 2020][8592523.870139] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:39 2020][8592523.875461] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:39 2020][8592523.880976] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.887946] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.895342] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:39 2020][8592523.901362] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:39 2020][8592523.907725] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:39 2020][8592523.914348] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:39 2020][8592523.921490] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:39 2020][8592523.926726] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:39 2020][8592523.932398] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.939797] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592523.946783] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.954100] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.960982] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592523.968302] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:39 2020][8592523.975185] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:39 2020][8592523.981402] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.988139] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592523.994788] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.001840] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:39 2020][8592524.007366] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.014468] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.022220] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:39 2020][8592524.029472] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.037333] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.044299] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:39 2020][8592524.049734] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.056208] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.063781] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:39 2020][8592524.068842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592524.075118] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:39 2020][8592524.081752] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592524.088021] ll_ost_io01_083 D ffff9c5a147d4100 0 73714 2 0x00000080 [Tue Mar 17 18:00:39 2020][8592524.095298] Call Trace: [Tue Mar 17 18:00:39 2020][8592524.097927] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:39 2020][8592524.104550] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:39 2020][8592524.110910] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:39 2020][8592524.116231] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:39 2020][8592524.121734] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.128708] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.136106] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:00:39 2020][8592524.142137] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:00:39 2020][8592524.148503] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:00:39 2020][8592524.155118] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:39 2020][8592524.160615] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:00:39 2020][8592524.165848] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:00:39 2020][8592524.172990] [] ? dqget+0x190/0x450 [Tue Mar 17 18:00:39 2020][8592524.178224] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:00:39 2020][8592524.183897] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.191297] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592524.198267] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.205598] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.212481] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.219794] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:00:39 2020][8592524.226676] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:39 2020][8592524.232893] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.239633] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.246282] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.253341] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:39 2020][8592524.258867] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.265955] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.273717] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:39 2020][8592524.280971] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.288832] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.295799] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:39 2020][8592524.301236] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.307709] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:39 2020][8592524.315280] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:39 2020][8592524.320342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592524.326632] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:39 2020][8592524.333245] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:39 2020][8592524.339513] ll_ost_io01_084 D ffff9c5a147d6180 0 73715 2 0x00000080 [Tue Mar 17 18:00:39 2020][8592524.346823] Call Trace: [Tue Mar 17 18:00:39 2020][8592524.349460] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.357122] [] schedule+0x29/0x70 [Tue Mar 17 18:00:39 2020][8592524.362296] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:39 2020][8592524.369353] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:39 2020][8592524.375368] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592524.382599] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:39 2020][8592524.389309] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.397059] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:39 2020][8592524.403263] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:39 2020][8592524.410055] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:39 2020][8592524.417373] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.424954] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.432095] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:40 2020][8592524.438368] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:40 2020][8592524.445163] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:40 2020][8592524.451379] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.458112] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.464760] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.471825] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:40 2020][8592524.477344] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.484425] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.492174] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:40 2020][8592524.499426] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.507286] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.514250] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:40 2020][8592524.519687] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.526162] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.533733] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:40 2020][8592524.538811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.545088] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:40 2020][8592524.551708] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.557984] ll_ost_io01_085 D ffff9c5a147d5140 0 73716 2 0x00000080 [Tue Mar 17 18:00:40 2020][8592524.565259] Call Trace: [Tue Mar 17 18:00:40 2020][8592524.567888] [] schedule+0x29/0x70 [Tue Mar 17 18:00:40 2020][8592524.573033] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.580090] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:40 2020][8592524.586101] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.593327] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:40 2020][8592524.599185] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.605900] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.613215] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:40 2020][8592524.619408] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.626205] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.633530] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.641112] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.648259] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:40 2020][8592524.655054] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:40 2020][8592524.661274] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.668026] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.674676] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.681728] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:40 2020][8592524.687252] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.694334] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.702088] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:40 2020][8592524.709341] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.717202] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.724167] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:40 2020][8592524.729605] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.736093] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.743666] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:40 2020][8592524.748720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.754997] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:40 2020][8592524.761616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.767890] ll_ost_io01_086 D ffff9c5a147d2080 0 73717 2 0x00000080 [Tue Mar 17 18:00:40 2020][8592524.775160] Call Trace: [Tue Mar 17 18:00:40 2020][8592524.777789] [] schedule+0x29/0x70 [Tue Mar 17 18:00:40 2020][8592524.782942] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.789991] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:40 2020][8592524.796025] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.803250] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:40 2020][8592524.809087] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.815793] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:40 2020][8592524.821983] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592524.828782] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.836105] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.843687] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.850828] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:40 2020][8592524.857621] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:40 2020][8592524.863853] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.870587] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.877235] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.884286] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:40 2020][8592524.889811] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.896892] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.904640] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:40 2020][8592524.911894] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.919752] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.926718] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:40 2020][8592524.932171] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.938646] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592524.946219] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:40 2020][8592524.951278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.957554] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:40 2020][8592524.964174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592524.970441] ll_ost_io01_087 D ffff9c4b6957b0c0 0 73718 2 0x00000080 [Tue Mar 17 18:00:40 2020][8592524.977717] Call Trace: [Tue Mar 17 18:00:40 2020][8592524.980356] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592524.988019] [] schedule+0x29/0x70 [Tue Mar 17 18:00:40 2020][8592524.993185] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.000237] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:40 2020][8592525.006255] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.013485] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.020193] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.027938] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:40 2020][8592525.034127] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.040926] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.048250] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.055833] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.062998] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:40 2020][8592525.069270] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:40 2020][8592525.076068] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:40 2020][8592525.082284] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.089023] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.095670] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.102722] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:40 2020][8592525.108763] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.116196] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.123277] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.131054] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:40 2020][8592525.138308] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.146171] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.153136] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:40 2020][8592525.158573] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.165049] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.172626] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:40 2020][8592525.177680] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592525.183955] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:40 2020][8592525.190576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592525.196866] ll_ost_io01_088 D ffff9c7a09e48000 0 73719 2 0x00000080 [Tue Mar 17 18:00:40 2020][8592525.204136] Call Trace: [Tue Mar 17 18:00:40 2020][8592525.206765] [] schedule+0x29/0x70 [Tue Mar 17 18:00:40 2020][8592525.211917] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.218968] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:40 2020][8592525.224987] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.232213] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:40 2020][8592525.238056] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.244769] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.252084] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:40 2020][8592525.258290] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:40 2020][8592525.265088] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.272406] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.279990] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:40 2020][8592525.287138] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:40 2020][8592525.293931] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:40 2020][8592525.300149] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.306886] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.313536] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.320604] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:40 2020][8592525.327934] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:40 2020][8592525.333695] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:40 2020][8592525.339217] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.346305] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.354055] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:40 2020][8592525.361313] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.369174] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.376140] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:40 2020][8592525.381577] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.388052] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:40 2020][8592525.395634] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:40 2020][8592525.400693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592525.406967] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:40 2020][8592525.413589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:40 2020][8592525.419863] ll_ost_io01_089 D ffff9c7a1ef30000 0 73720 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592525.427140] Call Trace: [Tue Mar 17 18:00:41 2020][8592525.429770] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592525.434922] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.441972] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592525.447991] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:41 2020][8592525.454194] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:41 2020][8592525.460990] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:41 2020][8592525.467209] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.473946] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.480595] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.487663] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:41 2020][8592525.494979] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:41 2020][8592525.500733] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:41 2020][8592525.506261] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.513347] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.521112] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:41 2020][8592525.528363] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.536225] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.543194] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:41 2020][8592525.548628] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.555102] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.562673] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:41 2020][8592525.567735] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592525.574010] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:41 2020][8592525.580630] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592525.586921] ll_ost_io01_090 D ffff9c7a1ef36180 0 73721 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592525.594190] Call Trace: [Tue Mar 17 18:00:41 2020][8592525.596821] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592525.601973] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.609024] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592525.615042] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.622272] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:41 2020][8592525.628110] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.634824] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.642138] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:41 2020][8592525.648344] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.655144] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.662462] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.670046] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.677190] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:41 2020][8592525.683984] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:41 2020][8592525.690203] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.696942] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.703590] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.710658] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:41 2020][8592525.717987] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:41 2020][8592525.723735] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:41 2020][8592525.729262] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.736344] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.744100] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:41 2020][8592525.751352] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.759211] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.766178] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:41 2020][8592525.771614] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.778089] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.785677] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:41 2020][8592525.790739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592525.797015] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:41 2020][8592525.803635] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592525.809910] ll_ost_io01_091 D ffff9c7a1ef34100 0 73722 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592525.817186] Call Trace: [Tue Mar 17 18:00:41 2020][8592525.819822] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.827487] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592525.832638] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.839688] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592525.845720] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.852952] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.859663] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.867413] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:41 2020][8592525.873603] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592525.880402] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.887726] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.895308] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592525.902447] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:41 2020][8592525.908724] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:41 2020][8592525.915533] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:41 2020][8592525.921753] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.928463] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:41 2020][8592525.934503] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.941938] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.949025] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.956780] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:41 2020][8592525.964031] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.971892] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.978860] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:41 2020][8592525.984313] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.990786] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592525.998358] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:41 2020][8592526.003420] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.009695] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:41 2020][8592526.016315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.022590] ll_ost_io02_049 D ffff9c7a1ef31040 0 73723 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592526.029859] Call Trace: [Tue Mar 17 18:00:41 2020][8592526.032488] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592526.037640] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.044706] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592526.050717] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.057942] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:41 2020][8592526.063786] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.070494] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:41 2020][8592526.076683] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.083480] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.090796] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.098379] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.105528] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:41 2020][8592526.112336] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:41 2020][8592526.118554] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.125283] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.132714] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.139796] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.147544] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:41 2020][8592526.154796] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.162657] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.169623] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:41 2020][8592526.175060] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.181544] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.189112] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:41 2020][8592526.194174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.200452] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:41 2020][8592526.207070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.213344] ll_ost_io02_050 D ffff9c8a1efaa080 0 73724 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592526.220613] Call Trace: [Tue Mar 17 18:00:41 2020][8592526.223242] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592526.228388] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.235445] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.242602] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592526.248614] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:00:41 2020][8592526.254803] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:00:41 2020][8592526.261598] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:41 2020][8592526.267815] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.274553] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.281201] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.288252] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:41 2020][8592526.293779] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.300858] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.308621] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:41 2020][8592526.315875] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.323738] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.330704] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:41 2020][8592526.336154] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.342630] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:41 2020][8592526.350218] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:41 2020][8592526.355271] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.361548] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:41 2020][8592526.368169] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:41 2020][8592526.374458] ll_ost_io01_092 D ffff9c8a33656180 0 73725 2 0x00000080 [Tue Mar 17 18:00:41 2020][8592526.381743] Call Trace: [Tue Mar 17 18:00:41 2020][8592526.384373] [] schedule+0x29/0x70 [Tue Mar 17 18:00:41 2020][8592526.389517] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.396568] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:41 2020][8592526.402585] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.409814] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:41 2020][8592526.416526] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:41 2020][8592526.424277] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:42 2020][8592526.430469] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:42 2020][8592526.437279] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.444596] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.452173] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.459322] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:42 2020][8592526.465595] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:00:42 2020][8592526.472130] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:00:42 2020][8592526.478434] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.485520] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.493269] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:42 2020][8592526.500521] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.508396] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.515365] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:42 2020][8592526.520800] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.527274] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.534845] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592526.539907] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.546173] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592526.552796] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.559071] ll_ost_io02_051 D ffff9c8a33655140 0 73726 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592526.566355] Call Trace: [Tue Mar 17 18:00:42 2020][8592526.568993] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.576656] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592526.581822] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:42 2020][8592526.588876] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:42 2020][8592526.594893] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:42 2020][8592526.602120] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:42 2020][8592526.608830] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.616573] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:42 2020][8592526.622764] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:42 2020][8592526.629561] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.636894] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.644468] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592526.651608] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:42 2020][8592526.657882] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:42 2020][8592526.664678] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:42 2020][8592526.670895] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.677635] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.684282] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.691335] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:42 2020][8592526.696859] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.703954] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.711705] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:42 2020][8592526.718958] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.726818] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.733783] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:42 2020][8592526.739219] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.745693] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.753265] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592526.758326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.764601] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592526.771237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.777504] ldlm_cn01_008 S ffff9c5991105140 0 73823 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592526.784773] Call Trace: [Tue Mar 17 18:00:42 2020][8592526.787404] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592526.793505] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592526.798683] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.805562] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592526.811344] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.817818] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.825390] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592526.830468] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.836744] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592526.843366] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.849639] ldlm_cn03_008 S ffff9c690a5cb0c0 0 74194 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592526.856916] Call Trace: [Tue Mar 17 18:00:42 2020][8592526.859544] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592526.865638] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592526.870815] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.877692] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592526.883476] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.889965] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.897539] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592526.902593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.908869] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592526.915488] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.921765] ldlm_cn03_009 S ffff9c8a327d4100 0 75136 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592526.929039] Call Trace: [Tue Mar 17 18:00:42 2020][8592526.931669] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592526.937771] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592526.942948] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.949841] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592526.955619] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.962093] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592526.969666] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592526.974726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.981002] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592526.987622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592526.993896] ll_ost_create03 S ffff9c79ce915140 0 75352 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592527.001172] Call Trace: [Tue Mar 17 18:00:42 2020][8592527.003801] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592527.009896] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592527.015093] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.021969] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592527.027752] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.034226] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.041798] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592527.046859] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.053134] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592527.059754] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.066030] ldlm_cb01_004 S ffff9c8a353130c0 0 76900 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592527.073323] Call Trace: [Tue Mar 17 18:00:42 2020][8592527.075952] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592527.082050] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592527.087222] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.094101] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592527.099886] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.106330] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:42 2020][8592527.112028] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.119600] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592527.124660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.130951] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592527.137564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.143840] ldlm_cb01_005 S ffff9c7a2b030000 0 76922 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592527.151117] Call Trace: [Tue Mar 17 18:00:42 2020][8592527.153744] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:42 2020][8592527.159840] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592527.165015] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.171895] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592527.177679] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.184124] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:42 2020][8592527.189835] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.197409] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592527.202471] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.208746] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592527.215366] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.221641] ldlm_bl_10 S ffff9c5a2862b0c0 0 78686 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592527.228915] Call Trace: [Tue Mar 17 18:00:42 2020][8592527.231547] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:42 2020][8592527.237406] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.244810] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592527.249998] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.257056] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:42 2020][8592527.262835] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:42 2020][8592527.270409] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:42 2020][8592527.275471] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.281746] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:42 2020][8592527.288367] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.294642] ldlm_cn03_010 D ffff9c6a22ab5140 0 79032 2 0x00000080 [Tue Mar 17 18:00:42 2020][8592527.301915] Call Trace: [Tue Mar 17 18:00:42 2020][8592527.304546] [] schedule+0x29/0x70 [Tue Mar 17 18:00:42 2020][8592527.309696] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:42 2020][8592527.316160] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:42 2020][8592527.322261] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:42 2020][8592527.328018] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:42 2020][8592527.333533] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.339981] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.346433] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.353870] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.360929] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.367899] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.375077] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.382398] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:42 2020][8592527.389467] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.396971] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.404036] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:42 2020][8592527.410069] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.417396] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:42 2020][8592527.424022] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:43 2020][8592527.430384] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:43 2020][8592527.436859] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.444207] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:43 2020][8592527.451544] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.458627] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.465619] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.472701] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.480563] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.487528] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:43 2020][8592527.492964] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.499439] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.507012] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.512088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.518365] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.524987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.531259] ll_ost_create02 S ffff9c79e44ee180 0 79128 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.538528] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.541157] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592527.547260] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.552436] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.559316] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592527.565099] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.571587] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.579161] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.584222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.590499] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.597117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.603384] ll_ost_create00 S ffff9c7a03b71040 0 80079 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.610661] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.613291] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592527.619391] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.624570] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.631464] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592527.637249] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.643695] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:43 2020][8592527.649390] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.656963] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.662024] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.668297] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.674919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.681195] ldlm_cn01_009 S ffff9c69b7074100 0 81389 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.688470] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.691099] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592527.697192] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.702387] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.709267] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:43 2020][8592527.715888] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592527.721671] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.728146] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.735718] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.740780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.747057] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.753690] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.759958] ldlm_cn00_008 S ffff9c8a35316180 0 88696 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.767234] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.769864] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592527.775966] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.781142] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.788021] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:43 2020][8592527.794642] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592527.800426] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.806900] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.814487] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.819540] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.825807] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.832428] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.838702] ldlm_cn02_009 S ffff9c5a2ab8a080 0 90301 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.845971] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.848602] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592527.854701] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.859880] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.866759] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:43 2020][8592527.873397] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592527.879179] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.885655] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592527.893227] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592527.898289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.904563] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592527.911185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592527.917458] ll_ost01_082 D ffff9c79ca368000 0 97756 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592527.924734] Call Trace: [Tue Mar 17 18:00:43 2020][8592527.927366] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:43 2020][8592527.932687] [] ? bh_lru_install+0x18a/0x1e0 [Tue Mar 17 18:00:43 2020][8592527.938714] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:43 2020][8592527.944029] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592527.949177] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:00:43 2020][8592527.955197] [] ? free_pcppages_bulk+0x17e/0x3a0 [Tue Mar 17 18:00:43 2020][8592527.961554] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:43 2020][8592527.966874] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:00:43 2020][8592527.973058] [] io_schedule+0x18/0x20 [Tue Mar 17 18:00:43 2020][8592527.978462] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:00:43 2020][8592527.983864] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:00:43 2020][8592527.989459] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:00:43 2020][8592527.994773] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:00:43 2020][8592528.001213] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.007319] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:00:43 2020][8592528.014203] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.022476] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:00:43 2020][8592528.030060] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.038328] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.045297] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.053049] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.059949] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.067091] [] evict+0xb4/0x180 [Tue Mar 17 18:00:43 2020][8592528.072064] [] iput+0xfc/0x190 [Tue Mar 17 18:00:43 2020][8592528.076957] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:00:43 2020][8592528.084284] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:00:43 2020][8592528.091692] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:00:43 2020][8592528.098853] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:00:43 2020][8592528.105479] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:00:43 2020][8592528.112210] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.119984] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.128008] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:43 2020][8592528.134489] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.141577] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.149322] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:43 2020][8592528.156576] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.164438] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.171402] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:43 2020][8592528.176837] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.183313] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.190886] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592528.195961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.202230] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592528.208851] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.215124] ll_ost01_083 D ffff9c79ca36b0c0 0 97757 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592528.222392] Call Trace: [Tue Mar 17 18:00:43 2020][8592528.225022] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:43 2020][8592528.231029] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592528.237650] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:43 2020][8592528.244013] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:43 2020][8592528.249369] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.256383] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.263466] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.271211] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:43 2020][8592528.278467] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.286325] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.293293] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:43 2020][8592528.298728] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.305201] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.312775] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592528.317833] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.324117] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592528.330739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.337020] ll_ost01_084 S ffff9c5e798930c0 0 97758 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592528.344290] Call Trace: [Tue Mar 17 18:00:43 2020][8592528.346920] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592528.353022] [] schedule+0x29/0x70 [Tue Mar 17 18:00:43 2020][8592528.358218] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.365094] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:43 2020][8592528.370878] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.377352] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:43 2020][8592528.384937] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:43 2020][8592528.389992] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.396260] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:43 2020][8592528.402880] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:43 2020][8592528.409147] ll_ost01_085 S ffff9c5e79892080 0 97759 2 0x00000080 [Tue Mar 17 18:00:43 2020][8592528.416414] Call Trace: [Tue Mar 17 18:00:43 2020][8592528.419043] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:43 2020][8592528.425138] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592528.430314] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.437193] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:44 2020][8592528.442991] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.449467] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.457040] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592528.462103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.468377] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592528.474998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.481271] ll_ost_io02_052 D ffff9c5e79895140 0 97760 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592528.488540] Call Trace: [Tue Mar 17 18:00:44 2020][8592528.491168] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592528.496312] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:44 2020][8592528.503380] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:44 2020][8592528.509398] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592528.516626] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:44 2020][8592528.522469] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:44 2020][8592528.529181] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.536496] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:44 2020][8592528.542688] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592528.549486] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.556810] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.564391] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.571547] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:44 2020][8592528.578342] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:44 2020][8592528.584562] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.591271] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:00:44 2020][8592528.596939] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:44 2020][8592528.602692] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:44 2020][8592528.608220] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.615308] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.623061] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:44 2020][8592528.630317] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.638191] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.645160] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:44 2020][8592528.650596] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.657074] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.664648] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592528.669704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.675978] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592528.682599] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.688877] ll_ost01_086 S ffff9c5e79890000 0 97761 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592528.696152] Call Trace: [Tue Mar 17 18:00:44 2020][8592528.698780] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:44 2020][8592528.704872] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592528.710058] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.716935] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:44 2020][8592528.722713] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.729187] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.736758] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592528.741820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.748095] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592528.754715] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592528.760997] ll_ost_io02_053 D ffff9c5e79891040 0 97762 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592528.768266] Call Trace: [Tue Mar 17 18:00:44 2020][8592528.770897] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592528.776047] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:00:44 2020][8592528.783542] [] ? mempool_alloc+0x6e/0x170 [Tue Mar 17 18:00:44 2020][8592528.789383] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:44 2020][8592528.795401] [] raid5_make_request+0x1a5/0xca0 [raid456] [Tue Mar 17 18:00:44 2020][8592528.802453] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:00:44 2020][8592528.807600] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:44 2020][8592528.813624] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.821475] [] md_handle_request+0xd0/0x150 [Tue Mar 17 18:00:44 2020][8592528.827493] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 18:00:44 2020][8592528.834631] [] md_make_request+0x79/0x190 [Tue Mar 17 18:00:44 2020][8592528.840474] [] generic_make_request+0x147/0x380 [Tue Mar 17 18:00:44 2020][8592528.846835] [] ? md_mergeable_bvec+0x46/0x50 [Tue Mar 17 18:00:44 2020][8592528.852935] [] submit_bio+0x70/0x150 [Tue Mar 17 18:00:44 2020][8592528.858360] [] ? lprocfs_oh_tally+0x17/0x40 [obdclass] [Tue Mar 17 18:00:44 2020][8592528.865336] [] osd_submit_bio+0x1c/0x60 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.872220] [] osd_do_bio.isra.35+0x9b5/0xab0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.879622] [] ? recalc_bh_state.part.21+0x70/0x90 [Tue Mar 17 18:00:44 2020][8592528.886262] [] ? jbd2_journal_get_write_access+0x32/0x40 [jbd2] [Tue Mar 17 18:00:44 2020][8592528.894014] [] osd_write_commit+0x3ec/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.901242] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592528.908562] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:44 2020][8592528.915358] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:44 2020][8592528.921575] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.928287] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:44 2020][8592528.933812] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.940899] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.948648] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:44 2020][8592528.955917] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.963777] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.970744] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:44 2020][8592528.976179] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.982654] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592528.990226] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592528.995288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.001561] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592529.008183] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.014457] ll_ost01_087 S ffff9c5e79896180 0 97763 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592529.021752] Call Trace: [Tue Mar 17 18:00:44 2020][8592529.024382] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:44 2020][8592529.030481] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592529.035660] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.042538] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:44 2020][8592529.048323] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.054796] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.062368] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592529.067430] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.073704] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592529.080338] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.086609] ll_ost_io02_054 D ffff9c5ed6750000 0 97764 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592529.093884] Call Trace: [Tue Mar 17 18:00:44 2020][8592529.096516] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592529.101665] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.108718] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:44 2020][8592529.114736] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.121960] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:44 2020][8592529.127805] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.134519] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.141848] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:44 2020][8592529.148032] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.154831] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.162154] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.169735] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.176878] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:44 2020][8592529.183671] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:44 2020][8592529.189888] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.196599] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:44 2020][8592529.202556] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.209204] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.216258] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:44 2020][8592529.221781] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.228869] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.236617] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:44 2020][8592529.243869] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.251730] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.258697] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:44 2020][8592529.264131] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.270606] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.278193] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:44 2020][8592529.283248] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.289522] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:44 2020][8592529.296144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:44 2020][8592529.302419] ll_ost_io02_055 D ffff9c5ed67530c0 0 97765 2 0x00000080 [Tue Mar 17 18:00:44 2020][8592529.309695] Call Trace: [Tue Mar 17 18:00:44 2020][8592529.312332] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.319996] [] schedule+0x29/0x70 [Tue Mar 17 18:00:44 2020][8592529.325147] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.332199] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:44 2020][8592529.338244] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.345471] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.352195] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.359959] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:44 2020][8592529.366148] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:44 2020][8592529.372946] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.380277] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.387851] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:44 2020][8592529.394992] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:44 2020][8592529.401280] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:44 2020][8592529.408095] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:44 2020][8592529.414313] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.421056] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:44 2020][8592529.427709] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.434759] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:45 2020][8592529.440287] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.447374] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.455124] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:45 2020][8592529.462375] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.470235] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.477218] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:45 2020][8592529.482655] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.489130] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.496700] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592529.501761] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.508037] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592529.514659] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.520933] ll_ost_io02_056 D ffff9c5ed6755140 0 97766 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592529.528208] Call Trace: [Tue Mar 17 18:00:45 2020][8592529.530848] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.538509] [] schedule+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592529.543676] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.550728] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:45 2020][8592529.556746] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.563975] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.570685] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.578428] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:45 2020][8592529.584619] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.591416] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.598733] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.606330] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.613472] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:45 2020][8592529.619747] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:45 2020][8592529.626541] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:45 2020][8592529.632757] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.639498] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.646146] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.653198] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:45 2020][8592529.658721] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.665803] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.673566] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:45 2020][8592529.680819] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.688679] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.695646] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:45 2020][8592529.701090] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.707563] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.715137] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592529.720198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.726473] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592529.733096] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.739384] ll_ost01_088 S ffff9c5ed6756180 0 97767 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592529.746653] Call Trace: [Tue Mar 17 18:00:45 2020][8592529.749284] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:45 2020][8592529.755385] [] schedule+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592529.760562] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.767441] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:45 2020][8592529.773223] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.779698] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.787271] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592529.792332] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.798621] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592529.805236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592529.811510] ll_ost01_089 D ffff9c5ed6752080 0 97768 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592529.818777] Call Trace: [Tue Mar 17 18:00:45 2020][8592529.821408] [] schedule+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592529.826560] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.833612] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:45 2020][8592529.839629] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.846855] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.853569] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.861075] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:45 2020][8592529.867266] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592529.874068] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.881388] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.888972] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592529.896119] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:45 2020][8592529.902393] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:45 2020][8592529.908494] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:45 2020][8592529.915226] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.923001] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.931028] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:45 2020][8592529.937506] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.944595] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.952349] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:45 2020][8592529.959603] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.967461] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.974429] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:45 2020][8592529.979865] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.986339] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592529.993912] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592529.998987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.005255] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592530.011877] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.018150] ll_ost_io03_046 D ffff9c5ed6754100 0 97769 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592530.025418] Call Trace: [Tue Mar 17 18:00:45 2020][8592530.028048] [] schedule+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592530.033201] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:45 2020][8592530.040251] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:45 2020][8592530.046269] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592530.053494] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:00:45 2020][8592530.059353] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:45 2020][8592530.066074] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.073385] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:45 2020][8592530.079575] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:45 2020][8592530.086373] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.093689] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.101272] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.108420] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:00:45 2020][8592530.115214] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:45 2020][8592530.121433] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.128159] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:45 2020][8592530.133683] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.140766] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.148522] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:45 2020][8592530.155775] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.163634] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.170599] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:45 2020][8592530.176035] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.182510] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.190082] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592530.195157] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.201426] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592530.208039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.214313] ll_ost01_090 S ffff9c5ed6751040 0 97770 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592530.221582] Call Trace: [Tue Mar 17 18:00:45 2020][8592530.224211] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:45 2020][8592530.230312] [] schedule+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592530.235482] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.242360] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:45 2020][8592530.248144] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.254630] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:45 2020][8592530.262200] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:45 2020][8592530.267258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.273533] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:45 2020][8592530.280156] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:45 2020][8592530.286429] ll_ost_io03_047 D ffff9c8a2fd0e180 0 97771 2 0x00000080 [Tue Mar 17 18:00:45 2020][8592530.293697] Call Trace: [Tue Mar 17 18:00:45 2020][8592530.296329] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:45 2020][8592530.301648] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:45 2020][8592530.308270] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:45 2020][8592530.314646] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:45 2020][8592530.319961] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:45 2020][8592530.325464] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.332441] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.339860] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:45 2020][8592530.346142] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.353034] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.361396] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.368633] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.376297] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:45 2020][8592530.381899] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.389562] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.397494] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.404902] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.411952] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:45 2020][8592530.417453] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:45 2020][8592530.425642] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.432879] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.440200] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:46 2020][8592530.446993] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:46 2020][8592530.453225] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.459958] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.466608] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.473659] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:46 2020][8592530.479696] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.487133] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.494220] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.501968] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:46 2020][8592530.509220] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.517084] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.524073] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:46 2020][8592530.529510] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.535983] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.543556] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592530.548617] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 18:00:46 2020][8592530.554889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.561157] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592530.567779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.574052] ll_ost_create02 S ffff9c7a25b0b0c0 0 98628 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592530.581327] Call Trace: [Tue Mar 17 18:00:46 2020][8592530.583960] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:46 2020][8592530.590063] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592530.595253] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.602133] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592530.607918] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.614389] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.621961] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592530.627017] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.633290] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592530.639911] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.646200] ll_ost02_079 D ffff9c7a25b0d140 0 98670 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592530.653480] Call Trace: [Tue Mar 17 18:00:46 2020][8592530.656110] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592530.661262] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:46 2020][8592530.668310] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:46 2020][8592530.674321] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:46 2020][8592530.681549] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:46 2020][8592530.688263] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.695751] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:46 2020][8592530.701942] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:46 2020][8592530.708754] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.716073] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.723655] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592530.730794] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:46 2020][8592530.737072] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:46 2020][8592530.743178] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:46 2020][8592530.749911] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.757683] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.765695] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:46 2020][8592530.772171] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.779266] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.787015] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:46 2020][8592530.794272] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.802128] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.809095] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592530.814879] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.821352] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.828923] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592530.833977] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.840252] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592530.846887] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.853155] ll_ost02_080 S ffff9c7a25b0e180 0 98671 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592530.860423] Call Trace: [Tue Mar 17 18:00:46 2020][8592530.863052] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:46 2020][8592530.869145] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592530.874322] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.881201] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592530.886984] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.893458] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.901030] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592530.906099] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.912367] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592530.918988] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592530.925264] ll_ost02_081 D ffff9c7a25b0c100 0 98672 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592530.932540] Call Trace: [Tue Mar 17 18:00:46 2020][8592530.935170] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:46 2020][8592530.941185] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592530.947806] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:46 2020][8592530.954168] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:46 2020][8592530.959521] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.966540] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.973618] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.981367] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:46 2020][8592530.988618] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592530.996480] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.003447] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:46 2020][8592531.008882] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.015356] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.022928] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592531.027990] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.034280] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592531.040894] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.047171] ll_ost02_082 S ffff9c7a25b0a080 0 98673 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592531.054446] Call Trace: [Tue Mar 17 18:00:46 2020][8592531.057074] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:46 2020][8592531.063169] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592531.068346] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.075224] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592531.081006] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.087481] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.095075] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592531.100131] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.106406] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592531.113028] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.119301] ll_ost02_083 S ffff9c7a38ac9040 0 98674 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592531.126570] Call Trace: [Tue Mar 17 18:00:46 2020][8592531.129199] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:46 2020][8592531.135291] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592531.140460] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.147340] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592531.153137] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.159614] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.167185] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592531.172239] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.178514] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592531.185137] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.191410] ll_ost02_084 S ffff9c79ce910000 0 98675 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592531.198686] Call Trace: [Tue Mar 17 18:00:46 2020][8592531.201314] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:46 2020][8592531.207408] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592531.212599] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.219481] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:46 2020][8592531.225265] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.231737] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.239309] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:46 2020][8592531.244363] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.250639] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:46 2020][8592531.257260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:46 2020][8592531.263535] ll_ost02_085 D ffff9c79ce916180 0 98676 2 0x00000080 [Tue Mar 17 18:00:46 2020][8592531.270810] Call Trace: [Tue Mar 17 18:00:46 2020][8592531.273439] [] schedule+0x29/0x70 [Tue Mar 17 18:00:46 2020][8592531.278583] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:46 2020][8592531.285651] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:46 2020][8592531.291669] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:46 2020][8592531.298898] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:46 2020][8592531.305611] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592531.313102] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:46 2020][8592531.319290] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:46 2020][8592531.326088] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592531.333413] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:46 2020][8592531.341024] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:46 2020][8592531.348169] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:46 2020][8592531.354445] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:46 2020][8592531.360563] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:46 2020][8592531.367309] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.375087] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.383092] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:46 2020][8592531.389571] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.396652] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.404401] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:46 2020][8592531.411659] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.419520] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:46 2020][8592531.426486] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:46 2020][8592531.431923] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.438397] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.445971] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.451030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.457307] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.463926] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.470199] ll_ost02_086 S ffff9c7a35a830c0 0 98677 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.477486] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.480115] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592531.486218] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.491394] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.498274] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592531.504056] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.510530] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.518102] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.523164] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.529439] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.536079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.542352] ll_ost02_087 S ffff9c55a2368000 0 98678 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.549627] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.552257] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592531.558360] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.563536] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.570415] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592531.576198] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.582671] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.590246] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.595320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.601589] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.608212] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.614486] ll_ost02_088 S ffff9c55a2369040 0 98679 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.621760] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.624389] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592531.630483] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.635660] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.642540] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592531.648321] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.654812] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.662385] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.667447] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.673724] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.680344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.686618] ll_ost02_089 S ffff9c6a05389040 0 98680 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.693894] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.696525] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592531.702625] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.707805] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.714694] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592531.720472] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.726947] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.734519] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.739581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.745856] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.752476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.758751] ll_ost02_090 D ffff9c6dcf4ae180 0 98681 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.766026] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.768655] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.773802] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:47 2020][8592531.780866] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:47 2020][8592531.786877] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:47 2020][8592531.794106] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:47 2020][8592531.800818] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:47 2020][8592531.808308] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:47 2020][8592531.814498] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:47 2020][8592531.821296] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:47 2020][8592531.828611] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:47 2020][8592531.836197] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:47 2020][8592531.843357] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:47 2020][8592531.849635] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:47 2020][8592531.855737] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:47 2020][8592531.862468] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.870240] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.878253] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:47 2020][8592531.884728] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.891808] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.899557] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:47 2020][8592531.906809] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.914685] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.921651] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:47 2020][8592531.927089] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.933560] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.941135] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592531.946196] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.952470] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592531.959090] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592531.965365] ll_ost02_091 S ffff9c6dcf4ac100 0 98682 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592531.972641] Call Trace: [Tue Mar 17 18:00:47 2020][8592531.975272] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592531.981373] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592531.986565] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592531.993445] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592531.999230] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.005703] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.013275] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592532.018336] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.024611] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592532.031233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.037521] ll_ost02_092 D ffff9c6dcf4a8000 0 98683 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592532.044791] Call Trace: [Tue Mar 17 18:00:47 2020][8592532.047423] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:47 2020][8592532.053436] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592532.060058] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:47 2020][8592532.066419] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:47 2020][8592532.071777] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.078777] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.085866] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.093623] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:47 2020][8592532.100889] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.108751] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.115715] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:47 2020][8592532.121154] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.127627] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.135200] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592532.140260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.146536] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592532.153157] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.159430] ll_ost02_093 S ffff9c7a35a85140 0 98684 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592532.166717] Call Trace: [Tue Mar 17 18:00:47 2020][8592532.169346] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:47 2020][8592532.175449] [] schedule+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592532.180625] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.187504] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592532.193286] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.199761] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.207332] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592532.212393] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.218668] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592532.225307] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.231580] ll_ost02_094 D ffff9c79ce911040 0 98685 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592532.238848] Call Trace: [Tue Mar 17 18:00:47 2020][8592532.241478] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:47 2020][8592532.247226] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:47 2020][8592532.253759] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592532.260383] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:47 2020][8592532.266741] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:47 2020][8592532.272102] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.279102] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.286202] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.293952] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:47 2020][8592532.301205] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.309066] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.316027] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:47 2020][8592532.321805] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.328280] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:47 2020][8592532.335852] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:47 2020][8592532.340914] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.347202] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:47 2020][8592532.353834] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:47 2020][8592532.360110] ll_ost02_095 D ffff9c79ce912080 0 98686 2 0x00000080 [Tue Mar 17 18:00:47 2020][8592532.367386] Call Trace: [Tue Mar 17 18:00:47 2020][8592532.370016] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:47 2020][8592532.376636] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:47 2020][8592532.382998] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:47 2020][8592532.388320] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:47 2020][8592532.393820] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:47 2020][8592532.400797] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:47 2020][8592532.408193] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 18:00:47 2020][8592532.414135] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 18:00:47 2020][8592532.420404] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:00:47 2020][8592532.425732] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 18:00:47 2020][8592532.432709] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.440205] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.447528] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.455188] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.462327] [] evict+0xb4/0x180 [Tue Mar 17 18:00:48 2020][8592532.467299] [] iput+0xfc/0x190 [Tue Mar 17 18:00:48 2020][8592532.472193] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.479539] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:00:48 2020][8592532.486947] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:00:48 2020][8592532.494107] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:00:48 2020][8592532.500730] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:00:48 2020][8592532.507464] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.515236] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.523247] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:48 2020][8592532.529726] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.536812] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.544559] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:48 2020][8592532.551829] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.559690] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.566657] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:48 2020][8592532.572439] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.578911] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.586483] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592532.591545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.597822] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592532.604442] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.610717] ll_ost02_096 D ffff9c79ce9130c0 0 98687 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592532.618009] Call Trace: [Tue Mar 17 18:00:48 2020][8592532.620640] [] schedule+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592532.625792] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:48 2020][8592532.632844] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:48 2020][8592532.638860] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592532.646088] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:48 2020][8592532.652803] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.660292] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:48 2020][8592532.666481] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592532.673279] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.680617] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.688194] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592532.695333] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:48 2020][8592532.701608] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:00:48 2020][8592532.707710] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:00:48 2020][8592532.714440] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.722213] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.730225] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:00:48 2020][8592532.736702] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.743784] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.751552] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:48 2020][8592532.758808] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.766669] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.773635] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:48 2020][8592532.779072] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.785543] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.793117] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592532.798178] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.804453] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592532.811077] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.817363] ll_ost02_097 S ffff9c79ce914100 0 98688 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592532.824634] Call Trace: [Tue Mar 17 18:00:48 2020][8592532.827262] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:48 2020][8592532.833355] [] schedule+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592532.838534] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.845411] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:48 2020][8592532.851196] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.857668] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.865242] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592532.870301] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.876593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592532.883215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.889490] ll_ost02_098 D ffff9c7a03b75140 0 98689 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592532.896768] Call Trace: [Tue Mar 17 18:00:48 2020][8592532.899395] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:48 2020][8592532.905403] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592532.912024] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:48 2020][8592532.918386] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:48 2020][8592532.923738] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.930733] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.937835] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.945586] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:48 2020][8592532.952839] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.960698] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.967663] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:48 2020][8592532.973104] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.979573] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592532.987147] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592532.992210] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592532.998483] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592533.005121] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592533.011396] ll_ost_io02_057 D ffff9c55a236b0c0 0 98928 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592533.018665] Call Trace: [Tue Mar 17 18:00:48 2020][8592533.021295] [] schedule+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592533.026446] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.033498] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:48 2020][8592533.039515] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.046741] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.053455] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.061206] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:48 2020][8592533.067409] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.074210] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.081527] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.089109] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.096259] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:48 2020][8592533.102532] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:48 2020][8592533.109326] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:48 2020][8592533.115543] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.122254] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:00:48 2020][8592533.127920] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:48 2020][8592533.133692] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:00:48 2020][8592533.140255] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.147687] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.154767] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.162514] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:48 2020][8592533.169768] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.177628] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.184593] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:48 2020][8592533.190029] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.196506] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.204093] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592533.209153] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592533.215428] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592533.222050] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592533.228327] ll_ost_io02_058 S ffff9c55a236e180 0 98929 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592533.235601] Call Trace: [Tue Mar 17 18:00:48 2020][8592533.238229] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:48 2020][8592533.244323] [] schedule+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592533.249501] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.256381] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:48 2020][8592533.262180] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.268652] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.276226] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:48 2020][8592533.281288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592533.287564] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:48 2020][8592533.294183] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:48 2020][8592533.300457] ll_ost_io02_059 D ffff9c55a236c100 0 98930 2 0x00000080 [Tue Mar 17 18:00:48 2020][8592533.307725] Call Trace: [Tue Mar 17 18:00:48 2020][8592533.310363] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.318026] [] schedule+0x29/0x70 [Tue Mar 17 18:00:48 2020][8592533.323192] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.330246] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:48 2020][8592533.336264] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.343490] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.350216] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.357962] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:48 2020][8592533.364154] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:48 2020][8592533.370964] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.378282] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.385860] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:48 2020][8592533.393021] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:48 2020][8592533.399298] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:48 2020][8592533.406093] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:48 2020][8592533.412309] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:48 2020][8592533.419019] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:00:48 2020][8592533.424688] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:48 2020][8592533.430442] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:49 2020][8592533.436396] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.443828] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.450910] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.458680] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:49 2020][8592533.465933] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.473795] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.480762] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:49 2020][8592533.486199] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.492672] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.500245] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592533.505305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592533.511581] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592533.518202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592533.524490] ll_ost_io02_060 D ffff9c55a236d140 0 98931 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592533.531761] Call Trace: [Tue Mar 17 18:00:49 2020][8592533.534390] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592533.541011] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:49 2020][8592533.547372] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:49 2020][8592533.552695] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:00:49 2020][8592533.558196] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.565172] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.572568] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:00:49 2020][8592533.578849] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.585753] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.594111] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.601348] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.609012] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:00:49 2020][8592533.614600] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.622269] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.630201] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.637608] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.644665] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.652852] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:49 2020][8592533.659836] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.667067] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.674387] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:00:49 2020][8592533.681182] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:49 2020][8592533.687400] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.694138] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.700789] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.707857] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:49 2020][8592533.715170] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:49 2020][8592533.720925] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:49 2020][8592533.726467] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.733547] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.741295] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:49 2020][8592533.748550] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.756409] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.763375] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:49 2020][8592533.768811] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.775284] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.782859] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592533.787919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592533.794209] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592533.800823] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592533.807097] ll_ost_io02_061 D ffff9c55a236a080 0 98932 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592533.814373] Call Trace: [Tue Mar 17 18:00:49 2020][8592533.817010] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.824674] [] schedule+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592533.829826] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:00:49 2020][8592533.836877] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:49 2020][8592533.842892] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:00:49 2020][8592533.850121] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:00:49 2020][8592533.856847] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.864591] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:00:49 2020][8592533.870784] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:00:49 2020][8592533.877582] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.884905] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.892487] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:00:49 2020][8592533.899627] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:00:49 2020][8592533.905902] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:00:49 2020][8592533.912698] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:00:49 2020][8592533.918913] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.925639] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:00:49 2020][8592533.931560] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:00:49 2020][8592533.937230] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:00:49 2020][8592533.942983] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:00:49 2020][8592533.948512] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.955598] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.963345] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:49 2020][8592533.970597] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.978459] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.985423] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:49 2020][8592533.990877] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592533.997351] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.004926] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.009987] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.016260] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.022882] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.029157] ll_ost_create02 D ffff9c6dcf4ad140 0 98979 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592534.036432] Call Trace: [Tue Mar 17 18:00:49 2020][8592534.039061] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:49 2020][8592534.045076] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592534.051710] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:00:49 2020][8592534.058068] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:00:49 2020][8592534.063424] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.070424] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.077504] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.085251] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:00:49 2020][8592534.092504] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.100366] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.107331] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:49 2020][8592534.112769] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.119256] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.126829] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.131883] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.138158] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.144781] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.151054] ldlm_bl_11 S ffff9c80cf9a9040 0 100092 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592534.158417] Call Trace: [Tue Mar 17 18:00:49 2020][8592534.161048] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:49 2020][8592534.166905] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:49 2020][8592534.174309] [] schedule+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592534.179497] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.186553] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:49 2020][8592534.192335] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.199909] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.204970] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.211244] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.217865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.224142] ldlm_bl_12 S ffff9c80cf9aa080 0 100093 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592534.231504] Call Trace: [Tue Mar 17 18:00:49 2020][8592534.234132] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:49 2020][8592534.239989] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:49 2020][8592534.247400] [] schedule+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592534.252567] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.259622] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:49 2020][8592534.265405] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.272979] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.278039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.284315] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.290935] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.297209] ldlm_bl_13 S ffff9c80cf9ab0c0 0 100094 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592534.304582] Call Trace: [Tue Mar 17 18:00:49 2020][8592534.307211] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:49 2020][8592534.313067] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:49 2020][8592534.320468] [] schedule+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592534.325638] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.332691] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:49 2020][8592534.338475] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.346062] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.351117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.357392] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.364041] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.370312] ldlm_bl_14 S ffff9c80cf9ac100 0 100095 2 0x00000080 [Tue Mar 17 18:00:49 2020][8592534.377682] Call Trace: [Tue Mar 17 18:00:49 2020][8592534.380313] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:49 2020][8592534.386165] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:49 2020][8592534.393567] [] schedule+0x29/0x70 [Tue Mar 17 18:00:49 2020][8592534.398743] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.405796] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:49 2020][8592534.411576] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:49 2020][8592534.419151] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:49 2020][8592534.424228] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:49 2020][8592534.430495] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:49 2020][8592534.437117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.443392] ldlm_bl_15 S ffff9c80cf9ad140 0 100096 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.450753] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.453384] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.459242] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.466646] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.471819] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.478874] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.484668] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.492236] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.497290] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.503565] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.510187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.516461] ldlm_bl_16 S ffff9c79cb1a5140 0 100097 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.523822] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.526452] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.532303] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.539704] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.544893] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.551943] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.557724] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.565300] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.570359] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.576634] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.583256] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.589530] ldlm_bl_17 S ffff9c79cb1a0000 0 100098 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.596893] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.599523] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.605379] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.612798] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.617966] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.625020] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.630802] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.638375] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.643436] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.649710] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.656331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.662599] ldlm_bl_18 S ffff9c79cb1a30c0 0 100099 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.669978] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.672610] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.678466] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.685869] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.691045] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.698100] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.703880] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.711452] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.716505] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.722779] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.729417] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.735684] ldlm_bl_19 S ffff9c79cb1a1040 0 100100 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.743038] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.745669] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.751528] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.758931] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.764110] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.771167] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.776947] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.784522] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.789599] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.795866] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.802480] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.808754] ldlm_bl_20 S ffff9c6e2197b0c0 0 100101 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.816107] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.818739] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.824595] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.832001] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.837172] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.844228] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.850023] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.857590] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.862645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.868918] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.875540] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.881814] ldlm_bl_21 S ffff9c6e2197e180 0 100102 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.889169] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.891797] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.897649] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.905050] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.910242] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.917296] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.923081] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.930649] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592534.935704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.941971] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592534.948593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592534.954868] ldlm_bl_22 S ffff9c6e21978000 0 100103 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592534.962230] Call Trace: [Tue Mar 17 18:00:50 2020][8592534.964859] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592534.970716] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592534.978136] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592534.983313] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592534.990365] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592534.996147] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.003723] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.008782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.015059] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.021678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.027952] ldlm_bl_23 S ffff9c6e2197c100 0 100104 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.035326] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.037955] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.043811] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.051208] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592535.056381] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.063436] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592535.069218] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.076790] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.081853] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.088127] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.094763] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.101031] ldlm_bl_24 S ffff9c6e2197a080 0 100105 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.108393] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.111024] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.116880] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.124276] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592535.129451] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.136503] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592535.142285] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.149861] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.154936] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.161204] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.167824] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.174092] ldlm_bl_25 S ffff9c6e2197d140 0 100106 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.181455] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.184084] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.189931] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.197326] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592535.202502] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.209557] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592535.215353] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.222930] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.227990] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.234266] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.240887] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.247161] ldlm_bl_26 S ffff9c6ae9a830c0 0 100107 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.254525] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.257154] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.263010] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.270414] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592535.275604] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.282659] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592535.288442] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.296017] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.301079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.307350] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.313972] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.320248] ldlm_bl_27 S ffff9c7a2c508000 0 100108 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.327609] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.330239] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.336090] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.343504] [] schedule+0x29/0x70 [Tue Mar 17 18:00:50 2020][8592535.348690] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.355748] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:50 2020][8592535.361528] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:50 2020][8592535.369103] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:50 2020][8592535.374162] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.380429] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:50 2020][8592535.387050] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:50 2020][8592535.393325] ldlm_bl_28 S ffff9c7a2c50b0c0 0 100109 2 0x00000080 [Tue Mar 17 18:00:50 2020][8592535.400706] Call Trace: [Tue Mar 17 18:00:50 2020][8592535.403335] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:50 2020][8592535.409192] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:50 2020][8592535.416594] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.421771] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.428824] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.434605] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.442179] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.447240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.453516] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.460152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.466420] ldlm_bl_29 S ffff9c7a2f61e180 0 100110 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.473775] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.476404] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.482262] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.489667] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.494839] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.501895] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.507675] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.515249] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.520323] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.526595] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.533215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.539488] ldlm_bl_30 S ffff9c7a09ed30c0 0 100111 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.546845] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.549473] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.555330] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.562724] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.567900] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.574953] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.580750] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.588317] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.593372] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.599646] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.606265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.612532] ldlm_bl_31 S ffff9c7a09ed0000 0 100112 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.619896] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.622525] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.628384] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.635786] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.640977] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.648032] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.653814] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.661387] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.666449] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.672724] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.679344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.685619] ldlm_bl_32 S ffff9c7a2c50c100 0 100113 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.692982] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.695611] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.701459] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.708871] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.714046] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.721102] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.726883] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.734456] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.739518] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.745794] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.752413] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.758688] ldlm_bl_33 S ffff9c7a2c50d140 0 100114 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.766067] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.768698] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.774555] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.781959] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.787133] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.794187] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.799968] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.807544] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.812604] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.818879] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.825516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.831782] ldlm_bl_34 S ffff9c7a2c509040 0 100115 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.839137] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.841768] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.847625] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.855028] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.860201] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.867256] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.873037] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.880612] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.885689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.891955] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.898570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.904845] ldlm_bl_35 S ffff9c7a2e331040 0 100116 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.912206] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.914837] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.920692] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592535.928089] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592535.933263] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.940316] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592535.946120] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592535.953689] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592535.958751] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.965025] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592535.971648] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592535.977920] ldlm_bl_36 S ffff9c7a37a2c100 0 100117 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592535.985276] Call Trace: [Tue Mar 17 18:00:51 2020][8592535.987904] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592535.993754] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.001158] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.006348] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.013404] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.019184] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.026760] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.031820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.038094] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.044716] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.050990] ldlm_bl_37 S ffff9c7a37a2a080 0 100118 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592536.058363] Call Trace: [Tue Mar 17 18:00:51 2020][8592536.060993] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592536.066851] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.074267] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.079436] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.086497] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.092273] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.099861] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.104915] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.111190] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.117813] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.124087] ldlm_bl_38 S ffff9c7a37a2b0c0 0 100119 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592536.131466] Call Trace: [Tue Mar 17 18:00:51 2020][8592536.134096] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592536.139953] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.147359] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.152532] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.159587] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.165368] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.172939] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.177993] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.184270] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.190905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.197172] ldlm_bl_39 S ffff9c7a37a28000 0 100120 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592536.204526] Call Trace: [Tue Mar 17 18:00:51 2020][8592536.207157] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592536.213014] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.220417] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.225592] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.232647] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.238428] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.246000] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.251070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.257346] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.263968] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.270242] ldlm_bl_40 S ffff9c7a2b031040 0 100121 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592536.277607] Call Trace: [Tue Mar 17 18:00:51 2020][8592536.280234] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592536.286083] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.293484] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.298652] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.305709] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.311503] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.319071] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.324131] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.330408] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.337029] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.343301] ldlm_bl_41 S ffff9c7a2b034100 0 100122 2 0x00000080 [Tue Mar 17 18:00:51 2020][8592536.350671] Call Trace: [Tue Mar 17 18:00:51 2020][8592536.353303] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:51 2020][8592536.359153] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:51 2020][8592536.366570] [] schedule+0x29/0x70 [Tue Mar 17 18:00:51 2020][8592536.371754] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.378810] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:51 2020][8592536.384590] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:51 2020][8592536.392165] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:51 2020][8592536.397227] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.403503] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:51 2020][8592536.410123] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:51 2020][8592536.416398] ldlm_bl_42 S ffff9c7a2b036180 0 100123 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.423761] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.426389] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.432238] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.439649] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.444816] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.451871] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.457653] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.465227] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.470303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.476571] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.483192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.489466] ldlm_bl_43 S ffff9c7a2b035140 0 100124 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.496839] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.499467] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.505317] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.512720] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.517895] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.524949] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.530730] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.538306] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.543365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.549642] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.556277] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.562546] ldlm_bl_44 S ffff9c7a2b0330c0 0 100125 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.569907] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.572537] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.578395] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.585797] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.590972] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.598027] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.603808] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.611382] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.616459] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.622728] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.629349] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.635623] ldlm_bl_45 S ffff9c7a03b74100 0 100126 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.642985] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.645614] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.651462] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.658858] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.664032] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.671091] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.676883] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.684453] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.689512] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.695790] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.702409] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.708683] ldlm_bl_46 S ffff9c7a03b76180 0 100127 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.716063] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.718697] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.724562] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.731963] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.737144] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.744214] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.749989] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.757563] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.762626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.768899] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.775523] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.781809] ldlm_bl_47 S ffff9c7a03b72080 0 100128 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.789168] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.791797] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.797656] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.805073] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.810242] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.817295] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.823077] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.830672] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.835731] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.842005] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.848626] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.854901] ldlm_bl_48 S ffff9c5a39d46180 0 100129 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.862280] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.864909] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.870773] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.878171] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.883359] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.890407] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.896206] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.903779] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.908858] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.915132] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.921769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.928038] ldlm_bl_49 S ffff9c5a39d41040 0 100130 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592536.935410] Call Trace: [Tue Mar 17 18:00:52 2020][8592536.938053] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592536.943907] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592536.951323] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592536.956492] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.963548] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592536.969328] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592536.976916] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592536.981972] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592536.988246] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592536.994867] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.001156] ldlm_bl_50 S ffff9c50359b30c0 0 100131 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.008512] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.011142] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.016992] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.024409] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.029578] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.036646] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.042441] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.050014] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.055074] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.061342] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592537.067963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.074253] ldlm_bl_51 S ffff9c50359b6180 0 100132 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.081608] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.084238] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.090096] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.097496] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.102680] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.109752] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.115526] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.123101] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.128163] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.134437] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592537.141057] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.147347] ldlm_bl_52 S ffff9c50359b1040 0 100133 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.154704] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.157333] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.163182] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.170600] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.175769] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.182824] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.188619] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.196188] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.201249] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.207522] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592537.214143] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.220418] ldlm_bl_53 S ffff9c50359b5140 0 100134 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.227799] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.230428] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.236280] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.243681] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.248856] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.255911] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.261691] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.269283] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.274342] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.280631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592537.287248] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.293523] ldlm_bl_54 S ffff9c5a3a3de180 0 100135 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.300885] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.303515] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.309372] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.316776] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.321951] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.329007] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.334787] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.342361] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.347437] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.353703] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:52 2020][8592537.360318] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:52 2020][8592537.366592] ldlm_bl_55 S ffff9c4b694e8000 0 100136 2 0x00000080 [Tue Mar 17 18:00:52 2020][8592537.373954] Call Trace: [Tue Mar 17 18:00:52 2020][8592537.376583] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:52 2020][8592537.382431] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:52 2020][8592537.389827] [] schedule+0x29/0x70 [Tue Mar 17 18:00:52 2020][8592537.395004] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.402057] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:52 2020][8592537.407839] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:52 2020][8592537.415412] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:52 2020][8592537.420472] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.426748] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.433369] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.439643] ldlm_bl_56 S ffff9c4b694eb0c0 0 100137 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.447007] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.449653] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.455512] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.462915] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.468105] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.475159] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.480933] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.488523] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.493585] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.499860] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.506483] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.512758] ldlm_bl_57 S ffff9c5a1dbd4100 0 100138 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.520119] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.522749] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.528606] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.536011] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.541186] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.548238] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.554020] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.561593] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.566655] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.572930] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.579552] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.585824] ldlm_bl_58 S ffff9c5a1dbd6180 0 100139 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.593181] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.595808] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.601658] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.609062] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.614252] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.621306] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.627079] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.634651] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.639704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.645974] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.652609] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.658877] ldlm_bl_59 S ffff9c5a1dbd5140 0 100140 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.666238] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.668869] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.674728] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.682130] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.687306] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.694360] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.700139] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.707716] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.712775] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.719052] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.725671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.731947] ldlm_bl_60 S ffff9c6a28ee1040 0 100141 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.739310] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.741938] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.747795] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.755190] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.760366] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.767421] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.773216] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.780792] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.785853] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.792144] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.798758] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.805033] ldlm_bl_61 S ffff9c6a28ee5140 0 100142 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.812395] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.815025] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.820882] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.828300] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.833482] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.840531] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.846314] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.853887] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.858949] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.865224] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.871845] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.878120] ldlm_bl_62 S ffff9c6a28ee0000 0 100143 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.885483] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.888110] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.893959] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.901371] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.906546] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.913601] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.919384] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.926955] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592537.932009] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.938283] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592537.944907] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592537.951181] ldlm_bl_63 S ffff9c69c06e0000 0 100144 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592537.958576] Call Trace: [Tue Mar 17 18:00:53 2020][8592537.961206] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592537.967055] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592537.974451] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592537.979625] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592537.986678] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592537.992461] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.000049] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.005103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.011378] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.018014] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.024283] ldlm_bl_64 S ffff9c69c06e6180 0 100145 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.031639] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.034267] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.040124] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.047519] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.052695] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.059749] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592538.065529] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.073105] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.078178] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.084448] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.091070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.097345] ldlm_bl_65 S ffff9c5ae945a080 0 100146 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.104708] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.107336] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.113185] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.120588] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.125780] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.132834] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592538.138615] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.146204] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.151260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.157536] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.164170] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.170439] ldlm_bl_66 S ffff9c609e298000 0 100147 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.177803] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.180432] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.186290] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.193692] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.198881] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.205945] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592538.211720] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.219295] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.224357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.230631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.237253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.243526] ldlm_bl_67 S ffff9c609e299040 0 100148 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.250890] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.253517] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.259368] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.266786] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.271953] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.279009] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592538.284790] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.292364] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.297425] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.303714] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.310330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.316605] ldlm_bl_68 S ffff9c609e29c100 0 100149 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.323984] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.326613] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.332461] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.339856] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.345032] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.352107] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:53 2020][8592538.357885] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:53 2020][8592538.365459] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:53 2020][8592538.370521] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.376796] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:53 2020][8592538.383431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:53 2020][8592538.389698] ldlm_bl_69 S ffff9c609e29d140 0 100150 2 0x00000080 [Tue Mar 17 18:00:53 2020][8592538.397071] Call Trace: [Tue Mar 17 18:00:53 2020][8592538.399698] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:53 2020][8592538.405550] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:53 2020][8592538.412953] [] schedule+0x29/0x70 [Tue Mar 17 18:00:53 2020][8592538.418126] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.425182] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.430962] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.438537] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.443612] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.449881] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.456502] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.462791] ldlm_bl_70 S ffff9c609e29a080 0 100151 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.470148] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.472777] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.478625] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.486021] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.491198] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.498252] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.504062] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.511631] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.516692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.522968] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.529589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.535864] ldlm_bl_71 S ffff9c6a3735c100 0 100152 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.543226] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.545856] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.551713] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.559116] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.564307] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.571362] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.577142] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.584719] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.589780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.596056] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.602675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.608950] ldlm_bl_72 S ffff9c6a1db8d140 0 100153 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.616313] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.618942] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.624815] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.632228] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.637403] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.644458] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.650240] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.657812] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.662873] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.669141] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.675762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.682036] ldlm_bl_73 S ffff9c5c85216180 0 100154 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.689417] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.692044] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.697894] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.705297] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.710472] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.717528] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.723307] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.730881] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.735945] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.742217] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.748854] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.755122] ldlm_bl_74 S ffff9c5c85211040 0 100155 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.762477] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.765122] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.772525] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.777703] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.784772] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.790553] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.798127] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.803189] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.809478] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.816097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.822383] ldlm_bl_75 S ffff9c5c85212080 0 100156 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.829739] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.832368] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.838217] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.845613] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.850787] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.857842] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.863622] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.871211] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.876265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.882533] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.889154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.895429] ldlm_bl_76 S ffff9c5c852130c0 0 100157 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.902792] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.905422] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.911278] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.918682] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.923856] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.930927] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592538.936707] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592538.944284] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592538.949343] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.955618] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592538.962239] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592538.968523] ldlm_bl_77 S ffff9c55c198c100 0 100158 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592538.975894] Call Trace: [Tue Mar 17 18:00:54 2020][8592538.978525] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592538.984382] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592538.991800] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592538.996968] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.004023] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.009819] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.017388] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.022449] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.028725] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.035345] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.041618] ldlm_bl_78 S ffff9c55c1989040 0 100159 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.048990] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.051620] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592539.057478] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.064897] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592539.070074] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.077128] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.082908] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.090481] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.095543] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.101817] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.108439] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.114728] ldlm_bl_79 S ffff9c55c1988000 0 100160 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.122085] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.124716] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592539.130590] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.137985] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592539.143159] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.150215] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.155995] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.163570] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.168631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.174933] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.181551] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.187827] ldlm_bl_80 S ffff9c55c198b0c0 0 100161 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.195190] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.197818] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592539.203667] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.211077] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592539.216246] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.223300] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.229080] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.236671] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.241725] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.248000] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.254622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.260897] ldlm_bl_81 S ffff9c55c198a080 0 100162 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.268258] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.270887] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592539.276738] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.284142] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592539.289316] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.296386] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.302166] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.309757] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.314811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.321077] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.327698] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.333973] ldlm_bl_82 S ffff9c55c198e180 0 100163 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.341337] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.343967] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:54 2020][8592539.349836] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.357263] [] schedule+0x29/0x70 [Tue Mar 17 18:00:54 2020][8592539.362438] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.369491] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:54 2020][8592539.375272] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:54 2020][8592539.382844] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:54 2020][8592539.387908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.394181] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:54 2020][8592539.400803] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:54 2020][8592539.407098] ldlm_bl_83 S ffff9c55c198d140 0 100164 2 0x00000080 [Tue Mar 17 18:00:54 2020][8592539.414457] Call Trace: [Tue Mar 17 18:00:54 2020][8592539.417105] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:54 2020][8592539.424519] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.429704] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.436763] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.442544] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.450118] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.455178] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.461454] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.468073] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.474348] ldlm_bl_84 S ffff9c7a39b60000 0 100165 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.481729] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.484374] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.490225] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.497627] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.502803] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.509858] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.515638] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.523228] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.528290] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.534567] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.541202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.547469] ldlm_bl_85 S ffff9c7a39b66180 0 100166 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.554824] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.557453] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.563309] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.570706] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.575880] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.582936] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.588715] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.596289] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.601366] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.607636] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.614255] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.620521] ldlm_bl_86 S ffff9c7a39b630c0 0 100167 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.627875] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.630506] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.636363] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.643783] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.648957] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.656013] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.661811] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.669384] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.674446] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.680722] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.687344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.693617] ldlm_bl_87 S ffff9c7a39b61040 0 100168 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.700979] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.703613] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.709479] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.716880] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.722063] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.729134] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.734920] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.742489] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.747549] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.753819] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.760437] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.766711] ldlm_bl_88 S ffff9c5c85215140 0 100169 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.774066] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.776700] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.782566] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.789966] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.795150] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.802206] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.807990] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.815569] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.820627] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.826903] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.833524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.839798] ldlm_bl_89 S ffff9c5c85214100 0 100170 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.847181] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.849811] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.855678] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.863078] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.868261] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.875316] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.881100] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.888670] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.893731] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.900007] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.906643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.912911] ldlm_bl_90 S ffff9c6a22792080 0 100171 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.920274] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.922904] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592539.928760] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592539.936156] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592539.941331] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.948385] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592539.954183] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592539.961756] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592539.966820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.973110] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592539.979733] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592539.986006] ldlm_bl_91 S ffff9c6a22794100 0 100172 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592539.993387] Call Trace: [Tue Mar 17 18:00:55 2020][8592539.996017] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.001871] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.009268] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.014442] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.021498] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.027294] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.034885] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.039938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.046215] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.052835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.059110] ldlm_bl_92 S ffff9c6a22796180 0 100173 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592540.066480] Call Trace: [Tue Mar 17 18:00:55 2020][8592540.069110] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.074959] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.082356] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.087545] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.094601] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.100383] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.107954] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.113024] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.119293] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.125913] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.132188] ldlm_bl_93 S ffff9c6d42618000 0 100174 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592540.139552] Call Trace: [Tue Mar 17 18:00:55 2020][8592540.142180] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.148030] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.155460] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.160634] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.167688] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.173469] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.181043] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.186105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.192394] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.199008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.205284] ldlm_bl_94 S ffff9c6d42619040 0 100175 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592540.212648] Call Trace: [Tue Mar 17 18:00:55 2020][8592540.215275] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.221132] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.228528] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.233703] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.240759] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.246537] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.254112] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.259174] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.265447] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.272103] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.278378] ldlm_bl_95 S ffff9c6d4261a080 0 100176 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592540.285740] Call Trace: [Tue Mar 17 18:00:55 2020][8592540.288370] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.294220] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.301623] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.306798] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.313866] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.319640] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.327217] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.332292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.338559] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.345185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.351469] ldlm_bl_96 S ffff9c6d4261b0c0 0 100177 2 0x00000080 [Tue Mar 17 18:00:55 2020][8592540.358841] Call Trace: [Tue Mar 17 18:00:55 2020][8592540.361474] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:55 2020][8592540.367323] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:55 2020][8592540.374726] [] schedule+0x29/0x70 [Tue Mar 17 18:00:55 2020][8592540.379901] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.386957] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:55 2020][8592540.392751] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:55 2020][8592540.400318] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:55 2020][8592540.405371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.411646] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:55 2020][8592540.418284] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:55 2020][8592540.424551] ldlm_bl_97 S ffff9c6d4261c100 0 100178 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.431915] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.434543] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:56 2020][8592540.440393] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:56 2020][8592540.447795] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.452987] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.460057] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592540.465832] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.473407] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592540.478468] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.484742] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592540.491365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.497637] ldlm_bl_98 S ffff9c6d4261d140 0 100179 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.504992] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.507622] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:56 2020][8592540.513479] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:56 2020][8592540.520897] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.526067] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.533121] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592540.538901] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.546475] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592540.551536] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.557811] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592540.564450] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.570723] ldlm_bl_99 S ffff9c6d4261e180 0 100180 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.578079] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.580722] [] ? call_rcu_sched+0x1d/0x20 [Tue Mar 17 18:00:56 2020][8592540.586575] [] ? obd_stale_export_get+0x31/0x190 [obdclass] [Tue Mar 17 18:00:56 2020][8592540.593977] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.599152] [] ldlm_bl_thread_main+0x715/0xa40 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.606222] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592540.611996] [] ? ldlm_handle_bl_callback+0x4f0/0x4f0 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.619570] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592540.624631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.630907] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592540.637542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.643826] bash S ffff9c7a35bbb0c0 0 102158 1 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.651181] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.653811] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.658955] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:00:56 2020][8592540.664188] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:00:56 2020][8592540.669508] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:00:56 2020][8592540.675610] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:00:56 2020][8592540.681798] journalctl S ffff9c6403f26180 0 102169 102158 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.689160] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.691789] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.696939] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 18:00:56 2020][8592540.704182] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 18:00:56 2020][8592540.710015] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:00:56 2020][8592540.716562] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 18:00:56 2020][8592540.722833] [] do_sys_poll+0x48d/0x590 [Tue Mar 17 18:00:56 2020][8592540.728410] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:00:56 2020][8592540.734427] [] ? tracing_is_on+0x15/0x30 [Tue Mar 17 18:00:56 2020][8592540.740179] [] ? tracing_record_cmdline+0x1d/0x120 [Tue Mar 17 18:00:56 2020][8592540.746801] [] ? probe_sched_wakeup+0x2b/0xa0 [Tue Mar 17 18:00:56 2020][8592540.753012] [] ? ttwu_do_wakeup+0xb5/0xe0 [Tue Mar 17 18:00:56 2020][8592540.758854] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:00:56 2020][8592540.764351] [] ? complete_walk+0x50/0xe0 [Tue Mar 17 18:00:56 2020][8592540.770104] [] ? default_wake_function+0x12/0x20 [Tue Mar 17 18:00:56 2020][8592540.776552] [] ? autoremove_wake_function+0x2b/0x40 [Tue Mar 17 18:00:56 2020][8592540.783261] [] ? update_time+0x89/0xd0 [Tue Mar 17 18:00:56 2020][8592540.788840] [] ? __sb_start_write+0x58/0x110 [Tue Mar 17 18:00:56 2020][8592540.794941] [] ? file_update_time+0xaa/0xf0 [Tue Mar 17 18:00:56 2020][8592540.800957] [] ? __sb_end_write+0x31/0x60 [Tue Mar 17 18:00:56 2020][8592540.806796] [] ? pipe_write+0x48f/0x540 [Tue Mar 17 18:00:56 2020][8592540.812470] [] ? do_sync_write+0x93/0xe0 [Tue Mar 17 18:00:56 2020][8592540.818220] [] SyS_ppoll+0x1d3/0x1f0 [Tue Mar 17 18:00:56 2020][8592540.823628] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:00:56 2020][8592540.829817] grep S ffff9c6403f20000 0 102170 102158 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.837179] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.839808] [] ? __sb_start_write+0x58/0x110 [Tue Mar 17 18:00:56 2020][8592540.845901] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.851050] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:00:56 2020][8592540.856284] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:00:56 2020][8592540.862301] [] pipe_read+0x298/0x400 [Tue Mar 17 18:00:56 2020][8592540.867736] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:00:56 2020][8592540.873227] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:00:56 2020][8592540.878463] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:00:56 2020][8592540.883611] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:00:56 2020][8592540.889799] ldlm_cb02_002 S ffff9c5a2862a080 0 102536 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.897160] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.899794] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592540.905907] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.911083] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.917967] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:56 2020][8592540.924599] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592540.930381] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.936824] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:56 2020][8592540.942521] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592540.950095] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592540.955152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.961421] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592540.968042] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592540.974315] ldlm_cb02_003 S ffff9c4f6e17a080 0 103707 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592540.981680] Call Trace: [Tue Mar 17 18:00:56 2020][8592540.984311] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592540.990412] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592540.995589] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.002467] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.008250] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.014695] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:56 2020][8592541.020408] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.027980] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.033034] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.039310] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.045945] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.052212] ldlm_cb02_004 S ffff9c4f6e17b0c0 0 103708 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592541.059585] Call Trace: [Tue Mar 17 18:00:56 2020][8592541.062215] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592541.068315] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592541.073495] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.080372] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.086154] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.092599] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:56 2020][8592541.098297] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.105896] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.110955] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.117222] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.123845] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.130117] ldlm_cn01_010 S ffff9c8a31a71040 0 105805 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592541.137472] Call Trace: [Tue Mar 17 18:00:56 2020][8592541.140103] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592541.146204] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592541.151382] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.158259] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:56 2020][8592541.164896] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.170688] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.177165] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.184735] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.189798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.196072] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.202692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.208981] ll_ost_create00 S ffff9c6fb712d140 0 108510 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592541.216339] Call Trace: [Tue Mar 17 18:00:56 2020][8592541.218969] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592541.225063] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592541.230256] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.237136] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.242920] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.249379] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:00:56 2020][8592541.255070] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.262642] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.267703] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.273978] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.280600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.286888] ldlm_cn01_011 S ffff9c8a393bd140 0 111297 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592541.294245] Call Trace: [Tue Mar 17 18:00:56 2020][8592541.296875] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592541.302976] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592541.308153] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.315032] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.320817] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.327291] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.334861] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.339938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.346219] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.352837] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.359127] ldlm_cn02_010 S ffff9c7a35bb8000 0 111298 2 0x00000080 [Tue Mar 17 18:00:56 2020][8592541.366496] Call Trace: [Tue Mar 17 18:00:56 2020][8592541.369131] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:56 2020][8592541.375232] [] schedule+0x29/0x70 [Tue Mar 17 18:00:56 2020][8592541.380423] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.387321] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:56 2020][8592541.393107] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.399584] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:56 2020][8592541.407174] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:56 2020][8592541.412230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:56 2020][8592541.418503] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:56 2020][8592541.425124] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.431390] ldlm_cn01_012 S ffff9c7a35bbe180 0 111299 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.438746] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.441375] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592541.447476] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.452653] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.459532] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592541.465330] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.471808] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.479380] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592541.484439] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.490715] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592541.497351] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.503619] ldlm_cn01_013 S ffff9c7a35bbc100 0 111300 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.510982] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.513612] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592541.519715] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.524906] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.531786] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:57 2020][8592541.538421] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592541.544200] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.550673] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.558247] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592541.563308] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.569582] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592541.576220] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.582485] ldlm_cn01_014 D ffff9c7a35bba080 0 111301 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.589850] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.592493] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.597636] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:57 2020][8592541.604085] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.610185] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:57 2020][8592541.615940] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:57 2020][8592541.621446] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.627902] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.634360] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.641767] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.648860] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.655831] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.662987] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.670304] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592541.677373] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.684878] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.691946] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:57 2020][8592541.697982] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.705312] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.711937] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:57 2020][8592541.718314] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:57 2020][8592541.724806] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.732140] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:57 2020][8592541.739479] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.746559] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.753550] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.760636] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.768494] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.775467] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:57 2020][8592541.780908] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.787394] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.794971] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592541.800030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.806323] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592541.812944] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.819220] ldlm_cn01_015 S ffff9c7a35bbd140 0 111302 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.826582] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.829214] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592541.835315] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.840505] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.847402] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592541.853187] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.859660] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.867234] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592541.872295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.878568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592541.885189] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.891464] ldlm_cn01_016 S ffff9c7a37a2d140 0 111303 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.898827] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.901456] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592541.907549] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.912743] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.919626] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:57 2020][8592541.926244] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592541.932028] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.938501] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592541.946075] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592541.951135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.957425] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592541.964040] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.970313] ldlm_cn00_009 D ffff9c7a37a2e180 0 111304 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592541.977687] Call Trace: [Tue Mar 17 18:00:57 2020][8592541.980315] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592541.985465] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:57 2020][8592541.991913] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592541.998029] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:57 2020][8592542.003776] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:57 2020][8592542.009273] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.015728] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.022186] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.029605] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.036677] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.043651] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.050811] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.058132] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:57 2020][8592542.065206] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.072712] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.079778] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:57 2020][8592542.085806] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.093139] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.099796] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:57 2020][8592542.106161] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:57 2020][8592542.112636] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.119963] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:57 2020][8592542.127305] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.134399] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.141386] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.148470] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.156332] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.163297] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:57 2020][8592542.168749] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.175241] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.182813] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592542.187876] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.194152] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592542.200772] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.207046] ldlm_cn01_017 S ffff9c79c98eb0c0 0 111305 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592542.214407] Call Trace: [Tue Mar 17 18:00:57 2020][8592542.217038] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592542.223139] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592542.228333] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.235214] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592542.240998] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.247486] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.255060] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592542.260122] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.266396] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592542.273017] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.279291] ldlm_cn00_010 S ffff9c79c98e8000 0 111306 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592542.286663] Call Trace: [Tue Mar 17 18:00:57 2020][8592542.289293] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592542.295393] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592542.300587] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.307470] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:57 2020][8592542.314090] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592542.319873] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.326346] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.333920] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592542.338979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.345255] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592542.351893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.358196] ldlm_cn00_011 S ffff9c79c98ea080 0 111307 2 0x00000080 [Tue Mar 17 18:00:57 2020][8592542.365573] Call Trace: [Tue Mar 17 18:00:57 2020][8592542.368203] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:57 2020][8592542.374307] [] schedule+0x29/0x70 [Tue Mar 17 18:00:57 2020][8592542.379483] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.386378] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:57 2020][8592542.393015] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:57 2020][8592542.398792] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.405265] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:57 2020][8592542.412852] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:57 2020][8592542.417908] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:57 2020][8592542.424182] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:57 2020][8592542.430818] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.437088] ldlm_cn01_018 S ffff9c5a39ea1040 0 111308 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.444449] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.447079] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592542.453180] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.458358] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.465238] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592542.471048] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.477518] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.485093] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592542.490154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.496428] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592542.503049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.509325] ldlm_cn00_012 S ffff9c5a39ea30c0 0 111309 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.516686] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.519317] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592542.525419] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.530626] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.537510] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:58 2020][8592542.544131] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592542.549914] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.556388] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.563960] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592542.569038] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.575311] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592542.581927] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.588199] ldlm_cn02_011 S ffff9c5a368f9040 0 111310 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.595580] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.598208] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592542.604303] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.609479] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.616360] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592542.622142] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.628617] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.636188] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592542.641250] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.647523] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592542.654160] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.660427] ldlm_cn01_019 D ffff9c5a39d40000 0 111311 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.667799] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.670431] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.675579] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:58 2020][8592542.682026] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.688126] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:58 2020][8592542.693883] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:58 2020][8592542.699384] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.705841] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.712328] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.719732] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.726788] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.733764] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.740917] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.748252] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:58 2020][8592542.755321] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.762826] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.769888] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:58 2020][8592542.775919] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.783268] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.789886] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:58 2020][8592542.796249] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:58 2020][8592542.802722] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.810049] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:58 2020][8592542.817400] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.824483] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.831477] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.838559] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.846421] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.853401] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:00:58 2020][8592542.858851] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.865330] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.872903] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592542.877963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.884238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592542.890861] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.897135] ldlm_cn01_020 S ffff9c5a39d44100 0 111312 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.904506] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.907136] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592542.913238] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.918429] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.925309] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592542.931094] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.937569] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.945141] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592542.950199] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.956482] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592542.963099] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592542.969379] ldlm_cn01_021 S ffff9c4b695c9040 0 111313 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592542.976752] Call Trace: [Tue Mar 17 18:00:58 2020][8592542.979381] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592542.985482] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592542.990659] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592542.997555] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.003339] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.009812] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.017386] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.022446] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.028720] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.035357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.041626] ldlm_cn01_022 S ffff9c5a3eff5140 0 111314 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.048996] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.051627] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592543.057729] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.062905] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.069786] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.075568] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.082042] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.089614] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.094692] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.100968] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.107605] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.113881] ldlm_cn01_023 S ffff9c5a162c0000 0 111315 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.121244] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.123871] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592543.129965] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.135143] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.142023] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:58 2020][8592543.148657] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.154450] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.160926] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.168498] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.173559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.179834] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.186471] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.192740] ldlm_cn01_024 S ffff9c5a162c2080 0 111316 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.200111] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.202739] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592543.208833] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.214026] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.220907] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.226689] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.233164] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.240736] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.245814] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.252090] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.258709] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.264984] ldlm_cn02_012 S ffff9c6a336e1040 0 111317 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.272346] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.274977] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592543.281079] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.286286] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.293160] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.298946] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.305417] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.312990] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.318052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.324327] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.330948] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.337235] ldlm_cn00_013 S ffff9c6a336e5140 0 111318 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.344593] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.347223] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:58 2020][8592543.353325] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.358523] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.365412] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:58 2020][8592543.371189] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.377663] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:58 2020][8592543.385237] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:58 2020][8592543.390297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.396588] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:58 2020][8592543.403204] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:58 2020][8592543.409491] ldlm_cn02_013 D ffff9c6a336e6180 0 111319 2 0x00000080 [Tue Mar 17 18:00:58 2020][8592543.416847] Call Trace: [Tue Mar 17 18:00:58 2020][8592543.419477] [] schedule+0x29/0x70 [Tue Mar 17 18:00:58 2020][8592543.424625] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:58 2020][8592543.431074] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.437176] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:59 2020][8592543.442929] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:59 2020][8592543.448445] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.454911] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.461365] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.468772] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.475836] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.482812] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.489975] [] ? cpumask_next_and+0x35/0x50 [Tue Mar 17 18:00:59 2020][8592543.496006] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.503326] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592543.510395] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.517898] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.524983] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.532308] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.538937] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:00:59 2020][8592543.545298] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:00:59 2020][8592543.551787] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.559116] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:00:59 2020][8592543.566457] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.573539] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.580532] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.587632] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.595508] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.602476] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.608260] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.614732] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.622308] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.627382] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.633650] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592543.640272] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.646546] ldlm_cn02_014 S ffff9c69c06e5140 0 111320 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592543.653909] Call Trace: [Tue Mar 17 18:00:59 2020][8592543.656539] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592543.662639] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592543.667832] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.674713] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.680498] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.686971] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.694560] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.699621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.705895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592543.712518] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.718806] ldlm_cn00_014 S ffff9c69c06e4100 0 111321 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592543.726163] Call Trace: [Tue Mar 17 18:00:59 2020][8592543.728792] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592543.734900] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592543.740070] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.746950] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.752736] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.759209] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.766781] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.771841] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.778131] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592543.784746] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.791021] ldlm_cn00_015 S ffff9c69c06e30c0 0 111322 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592543.798382] Call Trace: [Tue Mar 17 18:00:59 2020][8592543.801013] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592543.807115] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592543.812292] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.819169] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.824953] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.831447] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.839032] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.844088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.850363] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592543.856982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.863272] ldlm_cn02_015 S ffff9c6a2f211040 0 111323 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592543.870642] Call Trace: [Tue Mar 17 18:00:59 2020][8592543.873274] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592543.879368] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592543.884546] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.891425] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.897222] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.903698] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.911285] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.916341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.922617] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592543.929237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.935511] ldlm_cn01_025 S ffff9c69b7075140 0 111324 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592543.942874] Call Trace: [Tue Mar 17 18:00:59 2020][8592543.945504] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592543.951598] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592543.956791] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.963686] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592543.969463] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.975936] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592543.983509] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592543.988570] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592543.994844] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592544.001481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.007749] ldlm_cn01_026 S ffff9c69b7072080 0 111325 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592544.015113] Call Trace: [Tue Mar 17 18:00:59 2020][8592544.017740] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592544.023835] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592544.029026] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.035908] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592544.041704] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.048173] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.055744] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592544.060798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.067074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592544.073695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.079983] ldlm_cn01_027 S ffff9c5991102080 0 111455 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592544.087357] Call Trace: [Tue Mar 17 18:00:59 2020][8592544.089987] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592544.096080] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592544.101258] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.108137] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:59 2020][8592544.114758] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592544.120541] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.127015] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.134602] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592544.139671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.145942] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592544.152561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.158837] ldlm_cn00_016 S ffff9c5991106180 0 111456 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592544.166199] Call Trace: [Tue Mar 17 18:00:59 2020][8592544.168842] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592544.174940] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592544.180119] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.187002] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592544.192778] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.199253] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.206839] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592544.211895] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.218184] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592544.224800] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.231073] ldlm_cn01_028 S ffff9c7a32a8a080 0 111457 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592544.238429] Call Trace: [Tue Mar 17 18:00:59 2020][8592544.241056] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:00:59 2020][8592544.247150] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592544.252328] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.259233] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:00:59 2020][8592544.265852] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:00:59 2020][8592544.271629] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.278104] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:00:59 2020][8592544.285681] [] kthread+0xd1/0xe0 [Tue Mar 17 18:00:59 2020][8592544.290736] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.297009] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:00:59 2020][8592544.303631] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.309905] ldlm_cn00_017 D ffff9c7a32a8d140 0 111458 2 0x00000080 [Tue Mar 17 18:00:59 2020][8592544.317260] Call Trace: [Tue Mar 17 18:00:59 2020][8592544.319890] [] schedule+0x29/0x70 [Tue Mar 17 18:00:59 2020][8592544.325039] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:00:59 2020][8592544.331501] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:00:59 2020][8592544.337597] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:00:59 2020][8592544.343352] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:00:59 2020][8592544.348851] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.355316] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.361787] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.369192] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.376247] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.383229] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.390401] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:59 2020][8592544.397716] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:00:59 2020][8592544.404790] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:00:59 2020][8592544.412296] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:00:59 2020][8592544.419358] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:00:59 2020][8592544.425389] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:00:59 2020][8592544.432723] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:00 2020][8592544.439349] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:00 2020][8592544.445708] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:00 2020][8592544.452199] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.459526] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:00 2020][8592544.466863] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.473942] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.480936] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.488035] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.495897] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.502862] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:00 2020][8592544.508300] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.514772] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.522374] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.527433] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.533707] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.540329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.546603] ldlm_cn01_029 S ffff9c6b9c1b5140 0 111459 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.553967] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.556596] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.562713] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.567883] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.574762] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:00 2020][8592544.581397] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.587174] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.593650] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.601221] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.606283] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.612575] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.619195] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.625470] ldlm_cn01_030 S ffff9c6b9c1b6180 0 111460 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.632832] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.635462] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.641564] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.646758] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.653653] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.659429] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.665904] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.673477] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.678537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.684812] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.691432] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.697699] ldlm_cn01_031 S ffff9c6b9c1b1040 0 111461 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.705078] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.707709] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.713809] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.718987] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.725865] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.731649] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.738126] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.745695] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.750773] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.757040] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.763678] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.769952] ldlm_cn03_011 S ffff9c853672c100 0 122858 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.777307] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.779937] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.786038] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.791216] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.798095] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.803879] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.810352] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.817924] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.823000] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.829269] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.835890] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.842165] ldlm_cn03_012 S ffff9c62b0cf1040 0 128040 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.849528] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.852158] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.858258] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.863437] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.870314] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.876098] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.882603] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.890179] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.895240] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.901514] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.908136] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.914411] ldlm_cn00_018 S ffff9c70837a5140 0 30582 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.921688] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.924330] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592544.930426] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592544.935606] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.942499] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592544.948281] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.954756] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592544.962344] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592544.967398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.973673] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592544.980295] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592544.986569] ldlm_cn00_019 S ffff9c70837a30c0 0 30583 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592544.993845] Call Trace: [Tue Mar 17 18:01:00 2020][8592544.996474] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.002576] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.007786] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.014668] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.020451] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.026925] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.034497] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.039558] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.045848] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592545.052462] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.058729] ldlm_cn00_020 S ffff9c70837a2080 0 30584 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592545.066022] Call Trace: [Tue Mar 17 18:01:00 2020][8592545.068653] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.074754] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.079931] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.086810] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:00 2020][8592545.093431] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.099214] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.105688] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.113260] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.118338] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.124627] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592545.131244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.137517] ldlm_cn00_021 S ffff9c70837a4100 0 30585 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592545.144785] Call Trace: [Tue Mar 17 18:01:00 2020][8592545.147415] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.153518] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.158693] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.165572] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.171357] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.177829] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.185418] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.190481] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.196758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592545.203376] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.209653] ldlm_cn00_022 S ffff9c853672e180 0 30586 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592545.216927] Call Trace: [Tue Mar 17 18:01:00 2020][8592545.219558] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.225659] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.230852] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.237730] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.243507] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.249997] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.257568] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.262624] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.268913] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592545.275527] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.281801] ldlm_cn00_023 S ffff9c853672d140 0 30587 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592545.289070] Call Trace: [Tue Mar 17 18:01:00 2020][8592545.291698] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.297793] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.302998] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.309874] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.315656] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.322131] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.329700] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.334754] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.341021] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:00 2020][8592545.347658] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.353926] ldlm_cn00_024 S ffff9c8536729040 0 30588 2 0x00000080 [Tue Mar 17 18:01:00 2020][8592545.361203] Call Trace: [Tue Mar 17 18:01:00 2020][8592545.363831] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:00 2020][8592545.369964] [] schedule+0x29/0x70 [Tue Mar 17 18:01:00 2020][8592545.375159] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.382042] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:00 2020][8592545.387840] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.394315] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:00 2020][8592545.401887] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:00 2020][8592545.406951] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:00 2020][8592545.413238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592545.419852] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.426121] ldlm_cn00_025 S ffff9c8a24f42080 0 30589 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592545.433413] Call Trace: [Tue Mar 17 18:01:01 2020][8592545.436042] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592545.442136] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592545.447314] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.454192] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592545.459977] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.466450] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.474037] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592545.479090] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.485357] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592545.491980] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.498252] ldlm_cn00_026 S ffff9c8a24f430c0 0 30590 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592545.505521] Call Trace: [Tue Mar 17 18:01:01 2020][8592545.508149] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592545.514243] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592545.519422] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.526301] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592545.532081] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.538555] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.546143] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592545.551198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.557473] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592545.564095] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.570368] ldlm_cn00_027 D ffff9c8a24f46180 0 30591 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592545.577637] Call Trace: [Tue Mar 17 18:01:01 2020][8592545.580267] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592545.585415] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:01 2020][8592545.591866] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.597965] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:01 2020][8592545.603733] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:01 2020][8592545.609229] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.615685] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.622144] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.629553] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.636614] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.643583] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.650745] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.658066] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.665142] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.672646] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.679707] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:01:01 2020][8592545.685740] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.693074] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.699699] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:01 2020][8592545.706060] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:01 2020][8592545.712535] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.719863] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.727205] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.734285] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.741294] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.748376] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.756240] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.763204] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:01 2020][8592545.768642] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.775116] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.782685] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592545.787739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.794015] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592545.800634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.806916] ldlm_cn00_028 S ffff9c8a24f40000 0 30592 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592545.814186] Call Trace: [Tue Mar 17 18:01:01 2020][8592545.816816] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592545.822917] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592545.828095] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.834973] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:01 2020][8592545.841595] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592545.847379] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.853853] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592545.861424] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592545.866501] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.872777] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592545.879400] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.885673] ldlm_cn00_029 D ffff9c824ac00000 0 30593 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592545.892951] Call Trace: [Tue Mar 17 18:01:01 2020][8592545.895579] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592545.900727] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:01 2020][8592545.907176] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592545.913276] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:01 2020][8592545.919032] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:01 2020][8592545.924546] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.930997] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.937447] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.944856] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.951912] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.958886] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.966040] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.973360] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:01 2020][8592545.980429] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.987932] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:01 2020][8592545.995010] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:01:01 2020][8592546.001034] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:01 2020][8592546.008368] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:01 2020][8592546.014993] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:01 2020][8592546.021356] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:01 2020][8592546.027829] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.035157] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:01 2020][8592546.042499] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.049580] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.056573] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.063673] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.071534] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.078501] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:01 2020][8592546.083937] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.090411] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.097982] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592546.103042] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.109321] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592546.115939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.122215] ldlm_cn00_030 S ffff9c824ac02080 0 30594 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592546.129509] Call Trace: [Tue Mar 17 18:01:01 2020][8592546.132139] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592546.138238] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592546.143417] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.150295] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592546.156077] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.162552] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.170125] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592546.175185] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.181462] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592546.188097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.194363] ldlm_cn03_013 S ffff9c6b9c1b4100 0 31067 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592546.201631] Call Trace: [Tue Mar 17 18:01:01 2020][8592546.204264] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592546.210363] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592546.215541] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.222421] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592546.228201] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.234678] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.242251] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592546.247324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.253593] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592546.260215] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.266487] ldlm_cn02_016 S ffff9c6b9c1b0000 0 31068 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592546.273755] Call Trace: [Tue Mar 17 18:01:01 2020][8592546.276387] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592546.282487] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592546.287665] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.294544] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592546.300330] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.306801] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.314374] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592546.319452] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.325728] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592546.332364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.338638] ldlm_cn02_017 S ffff9c8a2538d140 0 31069 2 0x00000080 [Tue Mar 17 18:01:01 2020][8592546.345907] Call Trace: [Tue Mar 17 18:01:01 2020][8592546.348538] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:01 2020][8592546.354637] [] schedule+0x29/0x70 [Tue Mar 17 18:01:01 2020][8592546.359809] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.366686] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:01 2020][8592546.372485] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.378987] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:01 2020][8592546.386559] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:01 2020][8592546.391634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.397903] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:01 2020][8592546.404524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:01 2020][8592546.410799] ldlm_cn03_014 S ffff9c8a25388000 0 31070 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.418075] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.420704] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592546.426807] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.431984] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.438861] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592546.444658] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.451128] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.458698] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.463751] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.470019] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592546.476638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.482905] ldlm_cn02_018 S ffff9c8a33994100 0 31071 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.490173] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.492804] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592546.498905] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.504098] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.510977] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592546.516758] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.523233] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.530806] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.535858] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.542126] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592546.548746] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.555013] ldlm_cn02_019 S ffff9c80cf9a8000 0 31072 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.562298] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.564926] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592546.571020] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.576198] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.583075] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592546.588851] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.595323] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.602895] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.607949] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.614214] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592546.620842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.627111] ldlm_cn03_015 S ffff9c8a35314100 0 31073 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.634388] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.637016] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592546.643110] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.648278] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.655158] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592546.660941] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.667413] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.674987] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.680048] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.686338] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592546.692951] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.699219] ldlm_cn03_016 S ffff9c8a35315140 0 31074 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.706495] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.709123] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592546.715215] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.720386] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.727262] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592546.733037] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.739527] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.747095] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.752155] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.758430] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592546.765052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.771325] ldlm_cn03_017 D ffff9c7ae9fd2080 0 31075 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592546.778595] Call Trace: [Tue Mar 17 18:01:02 2020][8592546.781223] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592546.786371] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:02 2020][8592546.792811] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.798928] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:02 2020][8592546.804675] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:02 2020][8592546.810180] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.816637] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.823093] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.830499] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.837554] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.844520] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.851659] [] ? cpumask_next_and+0x35/0x50 [Tue Mar 17 18:01:02 2020][8592546.857699] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.865033] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592546.872104] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.879609] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.886679] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.894003] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.900629] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:02 2020][8592546.906990] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:02 2020][8592546.913465] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.920792] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:02 2020][8592546.928134] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.935228] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.942217] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.949298] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.957161] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.964127] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:02 2020][8592546.969563] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.976036] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592546.983607] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592546.988662] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592546.994937] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592547.001573] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.007841] ldlm_cn02_020 S ffff9c8a2e7b9040 0 31076 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592547.015124] Call Trace: [Tue Mar 17 18:01:02 2020][8592547.017757] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592547.023857] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592547.029035] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.035913] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:02 2020][8592547.042534] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592547.048317] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.054795] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.062362] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592547.067431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.073699] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592547.080320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.086585] ldlm_cn03_018 S ffff9c8a38f0a080 0 31077 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592547.093853] Call Trace: [Tue Mar 17 18:01:02 2020][8592547.096484] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592547.102585] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592547.107763] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.114640] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592547.120431] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.126906] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.134480] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592547.139540] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.145807] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592547.152427] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.158693] ldlm_cn02_021 S ffff9c8a3e8a0000 0 31078 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592547.165962] Call Trace: [Tue Mar 17 18:01:02 2020][8592547.168589] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592547.174682] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592547.179867] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.186747] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592547.192522] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.198998] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.206568] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592547.211621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.217895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592547.224510] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.230784] ldlm_cb02_005 S ffff9c896f8d8000 0 52157 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592547.238067] Call Trace: [Tue Mar 17 18:01:02 2020][8592547.240698] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:02 2020][8592547.246800] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592547.251991] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.258873] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:02 2020][8592547.264654] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.271100] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:01:02 2020][8592547.276788] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:02 2020][8592547.284363] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:02 2020][8592547.289422] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.295689] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:02 2020][8592547.302310] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.308585] ldlm_cn02_022 D ffff9c7f8cf71040 0 64115 2 0x00000080 [Tue Mar 17 18:01:02 2020][8592547.315860] Call Trace: [Tue Mar 17 18:01:02 2020][8592547.318490] [] schedule+0x29/0x70 [Tue Mar 17 18:01:02 2020][8592547.323629] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:02 2020][8592547.330071] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:02 2020][8592547.336171] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:02 2020][8592547.341916] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:02 2020][8592547.347411] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.353857] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.360306] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.367727] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.374804] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.381772] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.388940] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:02 2020][8592547.396254] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:02 2020][8592547.403321] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:02 2020][8592547.410825] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:02 2020][8592547.417885] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:01:03 2020][8592547.423912] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:03 2020][8592547.431236] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:03 2020][8592547.437878] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:03 2020][8592547.444240] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:03 2020][8592547.450717] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.458043] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:03 2020][8592547.465384] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.472465] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.479459] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.486542] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.494401] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.501368] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:03 2020][8592547.506818] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.513288] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.520859] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.525920] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.532195] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.538817] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.545089] ldlm_cn02_023 S ffff9c7f8cf730c0 0 64116 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.552358] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.554990] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.561090] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.566282] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.573161] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.578937] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.585414] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.592982] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.598036] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.604311] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.610932] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.617206] ldlm_cn02_024 S ffff9c7f8cf72080 0 64117 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.624493] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.627124] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.633223] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.638399] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.645278] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.651064] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.657536] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.665108] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.670169] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.676444] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.683065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.689345] ldlm_cn02_025 S ffff9c7f8cf75140 0 64118 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.696616] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.699246] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.705337] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.710507] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.717387] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:03 2020][8592547.724007] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.729791] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.736264] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.743835] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.748888] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.755154] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.761767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.768034] ldlm_cn02_026 S ffff9c535cc0e180 0 64119 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.775310] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.777939] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.784031] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.789202] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.796079] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.801868] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.808338] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.815907] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.820960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.827227] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.833840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.840105] ldlm_cn02_027 S ffff9c535cc0b0c0 0 64120 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.847376] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.850004] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.856105] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.861296] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.868169] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.873952] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.880427] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.887998] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.893049] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.899318] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.905938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.912204] ldlm_cn02_028 S ffff9c535cc0d140 0 64121 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.919472] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.922101] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592547.928195] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592547.933378] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.940261] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592547.946041] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.952516] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592547.960089] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592547.965148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.971416] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592547.978028] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592547.984295] ldlm_cn02_029 S ffff9c535cc0a080 0 64122 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592547.991570] Call Trace: [Tue Mar 17 18:01:03 2020][8592547.994201] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.000302] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.005472] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.012349] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.018133] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.024604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.032178] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.037230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.043513] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.050129] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.056403] ldlm_cn02_030 S ffff9c535cc08000 0 64123 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592548.063677] Call Trace: [Tue Mar 17 18:01:03 2020][8592548.066309] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.072409] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.077579] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.084455] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.090231] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.096704] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.104291] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.109346] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.115612] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.122226] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.128491] ldlm_cn02_031 S ffff9c535cc0c100 0 64124 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592548.135760] Call Trace: [Tue Mar 17 18:01:03 2020][8592548.138389] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.144480] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.149649] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.156530] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.162327] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.168802] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.176375] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.181427] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.187695] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.194315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.200582] ldlm_cn03_019 S ffff9c52e9249040 0 64125 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592548.207857] Call Trace: [Tue Mar 17 18:01:03 2020][8592548.210489] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.216589] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.221781] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.228661] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.234436] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.240910] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.248483] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.253545] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.259820] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.266441] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.272713] ldlm_cn03_020 S ffff9c5ab4666180 0 64126 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592548.279981] Call Trace: [Tue Mar 17 18:01:03 2020][8592548.282611] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.288705] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.293897] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.300778] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.306561] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.313036] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.320606] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.325662] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.331934] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.338547] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.344814] ldlm_cn03_021 S ffff9c5ab4660000 0 64127 2 0x00000080 [Tue Mar 17 18:01:03 2020][8592548.352089] Call Trace: [Tue Mar 17 18:01:03 2020][8592548.354738] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:03 2020][8592548.360837] [] schedule+0x29/0x70 [Tue Mar 17 18:01:03 2020][8592548.366008] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.372918] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:03 2020][8592548.378694] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.385169] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:03 2020][8592548.392739] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:03 2020][8592548.397793] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.404075] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:03 2020][8592548.410688] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:03 2020][8592548.416956] ldlm_cn03_022 S ffff9c5ab4665140 0 64128 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.424232] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.426858] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.432952] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.438122] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.444999] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.450775] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.457249] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.464838] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.469890] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.476156] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.482768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.489036] ldlm_cn03_023 S ffff9c5ab4662080 0 64129 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.496304] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.498932] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.505027] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.510204] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.517082] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.522879] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.529349] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.536920] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.541981] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.548247] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.554869] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.561143] ldlm_cn03_024 S ffff9c5ab4661040 0 64130 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.568411] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.571040] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.577132] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.582317] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.589196] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.594973] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.601448] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.609018] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.614071] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.620338] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.626958] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.633225] ldlm_cn03_025 S ffff9c5ab4664100 0 64131 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.640492] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.643122] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.649222] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.654406] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.661288] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:04 2020][8592548.667909] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.673692] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.680165] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.687739] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.692799] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.699066] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.705701] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.711971] ldlm_cn03_026 S ffff9c5ab46630c0 0 64132 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.719248] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.721875] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.727970] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.733147] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.740023] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:04 2020][8592548.746636] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.752416] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.758887] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.766473] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.771529] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.777802] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.784415] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.790683] ldlm_cn03_027 S ffff9c7ae98e30c0 0 64133 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.797958] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.800589] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.806690] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.811865] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.818744] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.824536] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.831011] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.838583] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.843643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.849909] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.856523] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.862796] ldlm_cn03_028 S ffff9c8a314d5140 0 64134 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.870067] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.872694] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.878787] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.883970] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.890843] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.896626] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.903099] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.910674] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.915734] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.922007] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592548.928621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.934894] ldlm_cn03_029 S ffff9c8a35312080 0 64135 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592548.942165] Call Trace: [Tue Mar 17 18:01:04 2020][8592548.944795] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592548.950895] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592548.956077] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.962959] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592548.968742] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.975215] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592548.982788] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592548.987842] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592548.994115] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592549.000729] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.007018] ldlm_cn03_030 S ffff9c7ae9fd4100 0 64136 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592549.014290] Call Trace: [Tue Mar 17 18:01:04 2020][8592549.016918] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592549.023019] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592549.028187] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.035066] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592549.040842] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.047316] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.054886] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592549.059938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.066221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592549.072835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.079102] ldlm_cn03_031 S ffff9c8a3ecab0c0 0 64137 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592549.086379] Call Trace: [Tue Mar 17 18:01:04 2020][8592549.089007] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:04 2020][8592549.095101] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592549.100277] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.107155] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:04 2020][8592549.112930] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.119404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.126977] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592549.132054] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.138323] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592549.144942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.151208] ll_ost_io03_048 D ffff9c6a0538b0c0 0 84741 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592549.158478] Call Trace: [Tue Mar 17 18:01:04 2020][8592549.161104] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592549.166258] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.173308] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:04 2020][8592549.179317] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.186560] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.193266] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:04 2020][8592549.201012] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:04 2020][8592549.207199] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.213997] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:04 2020][8592549.221315] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:04 2020][8592549.228894] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:04 2020][8592549.236034] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:04 2020][8592549.242309] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:04 2020][8592549.249104] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:04 2020][8592549.255343] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.262078] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.268726] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.275778] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:04 2020][8592549.281816] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.289242] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.296320] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.304071] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:04 2020][8592549.311321] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.319181] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.326147] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:04 2020][8592549.331597] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.338068] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:04 2020][8592549.345639] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:04 2020][8592549.350699] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.356989] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:04 2020][8592549.363606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:04 2020][8592549.369879] ll_ost_io03_049 D ffff9c757b806180 0 84742 2 0x00000080 [Tue Mar 17 18:01:04 2020][8592549.377184] Call Trace: [Tue Mar 17 18:01:04 2020][8592549.379812] [] schedule+0x29/0x70 [Tue Mar 17 18:01:04 2020][8592549.384961] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.392028] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:04 2020][8592549.398040] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.405268] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:04 2020][8592549.411980] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:04 2020][8592549.419721] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:05 2020][8592549.425903] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.432700] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.440017] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.447606] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.454763] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:05 2020][8592549.461039] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:05 2020][8592549.467835] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:05 2020][8592549.474055] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.480790] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.487438] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.494508] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:05 2020][8592549.501824] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:05 2020][8592549.507576] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:05 2020][8592549.514137] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.521570] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.528666] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.536416] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:05 2020][8592549.543668] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.551528] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.558493] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:05 2020][8592549.563931] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.570404] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.577975] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:05 2020][8592549.583027] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592549.589312] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:05 2020][8592549.595925] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592549.602198] ll_ost_io03_050 D ffff9c786ad6a080 0 84743 2 0x00000080 [Tue Mar 17 18:01:05 2020][8592549.609467] Call Trace: [Tue Mar 17 18:01:05 2020][8592549.612098] [] schedule+0x29/0x70 [Tue Mar 17 18:01:05 2020][8592549.617248] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.624299] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:05 2020][8592549.630308] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.637538] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:05 2020][8592549.643377] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.650106] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.657414] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:05 2020][8592549.663596] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.670396] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.677712] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.685282] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.692426] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:05 2020][8592549.699218] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:05 2020][8592549.705437] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.712167] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.718815] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.725879] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:05 2020][8592549.731401] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.738479] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.746228] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:05 2020][8592549.753480] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.761340] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.768309] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:05 2020][8592549.773743] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.780218] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.787803] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:05 2020][8592549.792859] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 18:01:05 2020][8592549.799135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592549.805409] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:05 2020][8592549.812031] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592549.818303] ll_ost_io03_051 D ffff9c786ad6d140 0 84744 2 0x00000080 [Tue Mar 17 18:01:05 2020][8592549.825571] Call Trace: [Tue Mar 17 18:01:05 2020][8592549.828201] [] schedule+0x29/0x70 [Tue Mar 17 18:01:05 2020][8592549.833343] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.840396] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:05 2020][8592549.846413] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.853656] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.860370] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.868113] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:05 2020][8592549.874303] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592549.881100] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.888416] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.895990] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592549.903140] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:05 2020][8592549.909412] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:05 2020][8592549.916208] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:05 2020][8592549.922439] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.929173] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.935821] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.942870] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:05 2020][8592549.948903] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.956330] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.963408] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.971155] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:05 2020][8592549.978407] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.986285] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592549.993252] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:05 2020][8592549.998687] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.005165] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.012733] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:05 2020][8592550.017785] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592550.024053] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:05 2020][8592550.030674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592550.036941] ll_ost_io03_052 D ffff9c786ad6c100 0 84745 2 0x00000080 [Tue Mar 17 18:01:05 2020][8592550.044216] Call Trace: [Tue Mar 17 18:01:05 2020][8592550.046845] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:05 2020][8592550.053458] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:05 2020][8592550.059827] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:05 2020][8592550.065144] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:05 2020][8592550.070642] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.077610] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.085005] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:05 2020][8592550.091278] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.098160] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.106516] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.113757] [] ? ldiskfs_es_insert_extent+0xdb/0x1c0 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.121420] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.128298] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:05 2020][8592550.133884] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.141546] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.149207] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.157131] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.164537] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.171593] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.179771] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592550.186758] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.193987] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.201299] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:05 2020][8592550.208094] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:05 2020][8592550.214311] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.221041] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.227692] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.234742] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:05 2020][8592550.240780] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.248206] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.255286] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.263050] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:05 2020][8592550.270303] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.278165] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.285134] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:05 2020][8592550.290567] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.297039] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:05 2020][8592550.304613] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:05 2020][8592550.309673] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592550.315940] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:05 2020][8592550.322576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:05 2020][8592550.328843] ll_ost_io00_053 D ffff9c5154109040 0 87179 2 0x00000080 [Tue Mar 17 18:01:05 2020][8592550.336113] Call Trace: [Tue Mar 17 18:01:05 2020][8592550.338741] [] schedule+0x29/0x70 [Tue Mar 17 18:01:05 2020][8592550.343883] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:05 2020][8592550.350935] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:05 2020][8592550.356944] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592550.364169] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:05 2020][8592550.370091] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:05 2020][8592550.376820] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.384130] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:05 2020][8592550.390332] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:05 2020][8592550.397126] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.404449] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.412037] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:05 2020][8592550.419182] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:06 2020][8592550.425976] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:06 2020][8592550.432194] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.438921] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:06 2020][8592550.446235] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:06 2020][8592550.451988] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:06 2020][8592550.457523] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.464606] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.472351] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:06 2020][8592550.479606] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.487464] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.494431] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:06 2020][8592550.499866] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.506342] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.513913] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:06 2020][8592550.518966] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.525256] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:06 2020][8592550.531871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.538144] ll_ost_io02_062 D ffff9c515410a080 0 87184 2 0x00000080 [Tue Mar 17 18:01:06 2020][8592550.545415] Call Trace: [Tue Mar 17 18:01:06 2020][8592550.548041] [] schedule+0x29/0x70 [Tue Mar 17 18:01:06 2020][8592550.553189] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.560244] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.567378] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:06 2020][8592550.573398] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:06 2020][8592550.579600] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:06 2020][8592550.586398] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:06 2020][8592550.592614] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.599344] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.605994] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.613045] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:06 2020][8592550.619084] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.626508] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.633588] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.641335] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:06 2020][8592550.648601] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.656457] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.663423] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:06 2020][8592550.668858] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.675333] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.682906] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:06 2020][8592550.687965] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.694232] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:06 2020][8592550.700844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.707110] ll_ost_io03_053 D ffff9c515410b0c0 0 87185 2 0x00000080 [Tue Mar 17 18:01:06 2020][8592550.714396] Call Trace: [Tue Mar 17 18:01:06 2020][8592550.717027] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:06 2020][8592550.723648] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:06 2020][8592550.730016] [] ? __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.738449] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:06 2020][8592550.743765] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:06 2020][8592550.749264] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.756234] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.763627] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:06 2020][8592550.769643] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:06 2020][8592550.776023] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:06 2020][8592550.782644] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:06 2020][8592550.789783] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:06 2020][8592550.795010] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:06 2020][8592550.800681] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.808082] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:06 2020][8592550.815054] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.822368] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.829250] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.836563] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:06 2020][8592550.843458] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:06 2020][8592550.849678] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.856380] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:06 2020][8592550.861906] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.868985] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.876733] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:06 2020][8592550.883987] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.891845] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.898812] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:06 2020][8592550.904248] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.910736] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592550.918302] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:06 2020][8592550.923357] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.929629] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:06 2020][8592550.936241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592550.942510] ll_ost_io02_063 D ffff9c515410c100 0 87186 2 0x00000080 [Tue Mar 17 18:01:06 2020][8592550.949787] Call Trace: [Tue Mar 17 18:01:06 2020][8592550.952413] [] schedule+0x29/0x70 [Tue Mar 17 18:01:06 2020][8592550.957556] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:06 2020][8592550.964609] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:06 2020][8592550.970627] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:06 2020][8592550.977865] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:06 2020][8592550.983704] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:06 2020][8592550.990419] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592550.997734] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:06 2020][8592551.003924] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:06 2020][8592551.010720] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.018037] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.025610] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.032751] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:06 2020][8592551.039561] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:06 2020][8592551.045781] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.052511] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.059161] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.066227] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:06 2020][8592551.073543] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:06 2020][8592551.079295] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:06 2020][8592551.084812] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.091892] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.099640] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:06 2020][8592551.106910] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.114771] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.121737] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:06 2020][8592551.127174] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.133648] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.141219] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:06 2020][8592551.146279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592551.152546] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:06 2020][8592551.159160] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592551.165434] ll_ost_io03_054 D ffff9c515410d140 0 87187 2 0x00000080 [Tue Mar 17 18:01:06 2020][8592551.172719] Call Trace: [Tue Mar 17 18:01:06 2020][8592551.175350] [] schedule+0x29/0x70 [Tue Mar 17 18:01:06 2020][8592551.180501] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:06 2020][8592551.187551] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:06 2020][8592551.193561] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:06 2020][8592551.200787] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:06 2020][8592551.206632] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:06 2020][8592551.213342] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.220650] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:06 2020][8592551.226841] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:06 2020][8592551.233655] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.240972] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.248545] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.255684] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:06 2020][8592551.262479] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:06 2020][8592551.268698] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.275398] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:06 2020][8592551.280926] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.288003] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.295753] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:06 2020][8592551.303021] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.310882] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.317849] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:06 2020][8592551.323285] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.329759] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:06 2020][8592551.337332] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:06 2020][8592551.342391] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592551.348658] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:06 2020][8592551.355280] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:06 2020][8592551.361555] ll_ost_io03_055 D ffff9c515410e180 0 87188 2 0x00000080 [Tue Mar 17 18:01:06 2020][8592551.368849] Call Trace: [Tue Mar 17 18:01:06 2020][8592551.371477] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:06 2020][8592551.378089] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:06 2020][8592551.384457] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:06 2020][8592551.389772] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:06 2020][8592551.395266] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.402234] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:06 2020][8592551.409628] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:06 2020][8592551.415641] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:06 2020][8592551.422001] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:07 2020][8592551.428632] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:07 2020][8592551.434123] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:07 2020][8592551.440482] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:07 2020][8592551.447624] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:07 2020][8592551.452858] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:07 2020][8592551.458527] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.465932] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:07 2020][8592551.472916] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.480233] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.487114] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.494442] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:07 2020][8592551.501327] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:07 2020][8592551.507546] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.514282] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.520930] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.527982] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:07 2020][8592551.533509] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.540587] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.548336] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:07 2020][8592551.555589] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.563464] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.570430] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:07 2020][8592551.575875] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.582349] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.589922] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:07 2020][8592551.594976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592551.601250] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:07 2020][8592551.607871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592551.614145] ll_ost_io02_064 D ffff9c79cab90000 0 87189 2 0x00000080 [Tue Mar 17 18:01:07 2020][8592551.621413] Call Trace: [Tue Mar 17 18:01:07 2020][8592551.624043] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:07 2020][8592551.630665] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:07 2020][8592551.637041] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:07 2020][8592551.642355] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:07 2020][8592551.647848] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.654815] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.662212] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:07 2020][8592551.668228] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:07 2020][8592551.674592] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:07 2020][8592551.681207] [] ? try_wait_for_completion+0x20/0x50 [Tue Mar 17 18:01:07 2020][8592551.687837] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:07 2020][8592551.694970] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:07 2020][8592551.700197] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:07 2020][8592551.705863] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.713262] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:07 2020][8592551.720231] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.727548] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.734428] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.741740] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:07 2020][8592551.748622] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:07 2020][8592551.754855] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.761586] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.768234] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.775285] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:07 2020][8592551.781318] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.788752] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.795830] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.803579] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:07 2020][8592551.810832] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.818692] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.825672] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:07 2020][8592551.831101] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.837578] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592551.845148] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:07 2020][8592551.850201] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592551.856469] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:07 2020][8592551.863089] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592551.869364] ll_ost_io02_065 D ffff9c79cab92080 0 87190 2 0x00000080 [Tue Mar 17 18:01:07 2020][8592551.876642] Call Trace: [Tue Mar 17 18:01:07 2020][8592551.879269] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:07 2020][8592551.885881] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:07 2020][8592551.892251] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:07 2020][8592551.897565] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:07 2020][8592551.903068] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.910034] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.917428] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:07 2020][8592551.923440] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:07 2020][8592551.929802] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:07 2020][8592551.936423] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.943125] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:07 2020][8592551.950273] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:07 2020][8592551.955500] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:07 2020][8592551.961165] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.968565] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:07 2020][8592551.975536] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.982850] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.989733] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592551.997044] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:07 2020][8592552.003926] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:07 2020][8592552.010145] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.016894] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.023540] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.030609] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:07 2020][8592552.037923] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:07 2020][8592552.043679] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:07 2020][8592552.050238] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.057671] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.064750] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.072497] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:07 2020][8592552.079751] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.087626] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.094595] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:07 2020][8592552.100031] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.106503] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.114076] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:07 2020][8592552.119133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592552.125413] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:07 2020][8592552.132026] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592552.138299] ll_ost_io03_056 D ffff9c79cab96180 0 87191 2 0x00000080 [Tue Mar 17 18:01:07 2020][8592552.145569] Call Trace: [Tue Mar 17 18:01:07 2020][8592552.148198] [] schedule+0x29/0x70 [Tue Mar 17 18:01:07 2020][8592552.153351] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592552.160423] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592552.167558] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:07 2020][8592552.173570] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:07 2020][8592552.179756] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:07 2020][8592552.186552] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:07 2020][8592552.192768] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.199499] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.206150] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.213215] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:07 2020][8592552.220521] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:07 2020][8592552.226267] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:07 2020][8592552.231786] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.238867] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.246614] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:07 2020][8592552.253866] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.261727] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.268692] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:07 2020][8592552.274129] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.280619] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:07 2020][8592552.288191] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:07 2020][8592552.293244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592552.299519] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:07 2020][8592552.306132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:07 2020][8592552.312404] ll_ost_io00_054 D ffff9c79cab94100 0 87192 2 0x00000080 [Tue Mar 17 18:01:07 2020][8592552.319676] Call Trace: [Tue Mar 17 18:01:07 2020][8592552.322303] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:07 2020][8592552.328926] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:07 2020][8592552.335286] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:07 2020][8592552.340614] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:07 2020][8592552.346110] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592552.353076] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:07 2020][8592552.360492] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:07 2020][8592552.366508] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:07 2020][8592552.372871] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:07 2020][8592552.379504] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:07 2020][8592552.384997] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:07 2020][8592552.390225] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:07 2020][8592552.397366] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:07 2020][8592552.402607] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:07 2020][8592552.408273] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:07 2020][8592552.415688] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:07 2020][8592552.422654] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.429967] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.436847] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.444163] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:08 2020][8592552.451044] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:08 2020][8592552.457261] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.463990] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.470638] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.477706] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:08 2020][8592552.483223] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.490304] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.498052] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:08 2020][8592552.505306] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.513172] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.520142] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:08 2020][8592552.525575] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.532050] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.539623] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:08 2020][8592552.544697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592552.550965] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:08 2020][8592552.557578] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592552.563845] ll_ost_io02_066 D ffff9c6ae98e8000 0 87193 2 0x00000080 [Tue Mar 17 18:01:08 2020][8592552.571113] Call Trace: [Tue Mar 17 18:01:08 2020][8592552.573750] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.581403] [] schedule+0x29/0x70 [Tue Mar 17 18:01:08 2020][8592552.586547] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:08 2020][8592552.593600] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:08 2020][8592552.599617] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592552.606860] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:08 2020][8592552.613572] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.621317] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:08 2020][8592552.627505] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592552.634302] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.641621] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.649202] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.656341] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:08 2020][8592552.662616] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:08 2020][8592552.669425] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:08 2020][8592552.675645] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.682375] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.689023] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.696092] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:08 2020][8592552.703406] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:08 2020][8592552.709153] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:08 2020][8592552.715713] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.723148] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.730233] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.737996] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:08 2020][8592552.745251] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.753112] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.760078] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:08 2020][8592552.765515] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.771990] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.779560] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:08 2020][8592552.784613] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592552.790880] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:08 2020][8592552.797501] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592552.803788] ll_ost_io03_057 D ffff9c4e3937e180 0 87194 2 0x00000080 [Tue Mar 17 18:01:08 2020][8592552.811063] Call Trace: [Tue Mar 17 18:01:08 2020][8592552.813691] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:08 2020][8592552.820312] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:08 2020][8592552.826673] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:08 2020][8592552.831985] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:08 2020][8592552.837478] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.844446] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.851841] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:08 2020][8592552.858122] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.866489] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:08 2020][8592552.872857] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.880084] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.887738] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:08 2020][8592552.893317] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.900982] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.908910] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.916310] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.923369] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.931553] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592552.938539] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.945768] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592552.953080] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:08 2020][8592552.959876] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:08 2020][8592552.966094] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.972820] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.980252] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.987333] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592552.995080] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:08 2020][8592553.002335] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.010208] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.017168] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:08 2020][8592553.022603] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.029080] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.036651] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:08 2020][8592553.041711] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592553.047977] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:08 2020][8592553.054590] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592553.060857] ll_ost_io02_067 D ffff9c4e39378000 0 87195 2 0x00000080 [Tue Mar 17 18:01:08 2020][8592553.068132] Call Trace: [Tue Mar 17 18:01:08 2020][8592553.070771] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.078432] [] schedule+0x29/0x70 [Tue Mar 17 18:01:08 2020][8592553.083591] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.090644] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:08 2020][8592553.096652] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.103882] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.110590] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.118336] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:08 2020][8592553.124525] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.131322] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.138637] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.146226] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.153370] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:08 2020][8592553.159644] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:08 2020][8592553.166439] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:08 2020][8592553.172656] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.179387] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.186034] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.193085] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:08 2020][8592553.198613] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.205705] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.213455] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:08 2020][8592553.220710] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.228569] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.235534] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:08 2020][8592553.240972] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.247446] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.255017] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:08 2020][8592553.260069] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592553.266335] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:08 2020][8592553.272963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:08 2020][8592553.279232] ll_ost_io03_058 D ffff9c4e39379040 0 87196 2 0x00000080 [Tue Mar 17 18:01:08 2020][8592553.286500] Call Trace: [Tue Mar 17 18:01:08 2020][8592553.289129] [] schedule+0x29/0x70 [Tue Mar 17 18:01:08 2020][8592553.294282] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.301332] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:08 2020][8592553.307349] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.314577] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.321287] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.329031] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:08 2020][8592553.335226] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:08 2020][8592553.342021] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.349336] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.356909] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:08 2020][8592553.364068] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:08 2020][8592553.370340] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:08 2020][8592553.377135] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:08 2020][8592553.383368] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.390101] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.396748] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:08 2020][8592553.403813] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:08 2020][8592553.409475] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:08 2020][8592553.415221] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:08 2020][8592553.421774] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.429206] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.436287] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.444034] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:09 2020][8592553.451288] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.459150] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.466119] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:09 2020][8592553.471564] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.478041] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.485613] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:09 2020][8592553.490674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.496941] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:09 2020][8592553.503563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.509837] ll_ost_io03_059 D ffff9c4e3937d140 0 87197 2 0x00000080 [Tue Mar 17 18:01:09 2020][8592553.517112] Call Trace: [Tue Mar 17 18:01:09 2020][8592553.519742] [] schedule+0x29/0x70 [Tue Mar 17 18:01:09 2020][8592553.524886] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.531958] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.539096] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:09 2020][8592553.545113] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:09 2020][8592553.551303] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:09 2020][8592553.558096] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:09 2020][8592553.564313] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.571016] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:09 2020][8592553.576541] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.583622] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.591368] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:09 2020][8592553.598637] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.606500] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.613465] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:09 2020][8592553.618901] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.625377] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.632946] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:09 2020][8592553.638000] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.644275] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:09 2020][8592553.650895] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.657162] ll_ost_io03_060 D ffff9c4e3937c100 0 87198 2 0x00000080 [Tue Mar 17 18:01:09 2020][8592553.664447] Call Trace: [Tue Mar 17 18:01:09 2020][8592553.667085] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.674737] [] schedule+0x29/0x70 [Tue Mar 17 18:01:09 2020][8592553.679881] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:09 2020][8592553.686932] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:09 2020][8592553.692941] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:09 2020][8592553.700170] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:09 2020][8592553.706880] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.714625] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:09 2020][8592553.720814] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:09 2020][8592553.727627] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.734943] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.742519] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.749658] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:09 2020][8592553.755932] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:09 2020][8592553.762727] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:09 2020][8592553.768942] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.775646] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:09 2020][8592553.781174] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.788260] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.796022] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:09 2020][8592553.803278] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.811140] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.818111] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:09 2020][8592553.823539] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.830012] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.837585] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:09 2020][8592553.842639] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.848914] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:09 2020][8592553.855535] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592553.861824] ll_ost_io03_061 D ffff9c68c37c4100 0 87199 2 0x00000080 [Tue Mar 17 18:01:09 2020][8592553.869095] Call Trace: [Tue Mar 17 18:01:09 2020][8592553.871726] [] schedule+0x29/0x70 [Tue Mar 17 18:01:09 2020][8592553.876877] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.883933] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592553.891069] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:09 2020][8592553.897085] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:09 2020][8592553.903274] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:09 2020][8592553.910068] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:09 2020][8592553.916287] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.923004] [] ? ___slab_alloc+0x24e/0x4f0 [Tue Mar 17 18:01:09 2020][8592553.928954] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.935603] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.942653] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:09 2020][8592553.948694] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.956128] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.963205] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.970955] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:09 2020][8592553.978206] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.986068] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592553.993046] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:09 2020][8592553.998478] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.004952] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.012526] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:09 2020][8592554.017587] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592554.023860] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:09 2020][8592554.030483] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592554.036755] ll_ost_io00_055 D ffff9c68c37c5140 0 87200 2 0x00000080 [Tue Mar 17 18:01:09 2020][8592554.044023] Call Trace: [Tue Mar 17 18:01:09 2020][8592554.046653] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:09 2020][8592554.053267] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:09 2020][8592554.059644] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:09 2020][8592554.064958] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:09 2020][8592554.070460] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.077426] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.084821] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:09 2020][8592554.090830] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:09 2020][8592554.097194] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:09 2020][8592554.103810] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:09 2020][8592554.110942] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:09 2020][8592554.116183] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:09 2020][8592554.121851] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.129251] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:09 2020][8592554.136222] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.143536] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.150417] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.157730] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:09 2020][8592554.164612] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:09 2020][8592554.170827] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.177559] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.184222] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.191277] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:09 2020][8592554.196802] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.203883] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.211628] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:09 2020][8592554.218883] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.226743] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.233708] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:09 2020][8592554.239146] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.245616] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.253191] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:09 2020][8592554.258251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592554.264517] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:09 2020][8592554.271131] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:09 2020][8592554.277405] ll_ost_io00_056 D ffff9c68c37c1040 0 87201 2 0x00000080 [Tue Mar 17 18:01:09 2020][8592554.284674] Call Trace: [Tue Mar 17 18:01:09 2020][8592554.287304] [] schedule+0x29/0x70 [Tue Mar 17 18:01:09 2020][8592554.292453] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:09 2020][8592554.299505] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:09 2020][8592554.305524] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:09 2020][8592554.312763] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:09 2020][8592554.318601] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:09 2020][8592554.325315] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.332631] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:09 2020][8592554.338819] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:09 2020][8592554.345617] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.352933] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.360506] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:09 2020][8592554.367668] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:09 2020][8592554.374459] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:09 2020][8592554.380678] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.387444] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.394087] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.401178] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:09 2020][8592554.408490] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:09 2020][8592554.414244] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:09 2020][8592554.419771] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:09 2020][8592554.426851] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.434597] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:10 2020][8592554.441849] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.449728] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.456694] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:10 2020][8592554.462131] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.468604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.476176] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:10 2020][8592554.481227] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.487495] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:10 2020][8592554.494118] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.500391] ll_ost_io03_062 D ffff9c68c37c30c0 0 87202 2 0x00000080 [Tue Mar 17 18:01:10 2020][8592554.507660] Call Trace: [Tue Mar 17 18:01:10 2020][8592554.510297] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.517951] [] schedule+0x29/0x70 [Tue Mar 17 18:01:10 2020][8592554.523116] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.530170] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:10 2020][8592554.536187] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.543417] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.550128] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.557868] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:10 2020][8592554.564050] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.570849] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.578166] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.585754] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.592898] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:10 2020][8592554.599168] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:10 2020][8592554.605965] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:10 2020][8592554.612185] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.618921] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.625568] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.632620] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:10 2020][8592554.638149] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.645248] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.653000] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:10 2020][8592554.660253] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.668114] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.675077] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:10 2020][8592554.680505] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.686979] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.694551] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:10 2020][8592554.699604] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.705872] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:10 2020][8592554.712508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.718775] ll_ost_io03_063 D ffff9c68c37c2080 0 87203 2 0x00000080 [Tue Mar 17 18:01:10 2020][8592554.726042] Call Trace: [Tue Mar 17 18:01:10 2020][8592554.728675] [] schedule+0x29/0x70 [Tue Mar 17 18:01:10 2020][8592554.733825] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.740875] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:10 2020][8592554.746884] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.754113] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.760824] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.768566] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:10 2020][8592554.774762] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.781554] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.788869] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.796443] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.803583] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:10 2020][8592554.809859] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:10 2020][8592554.816652] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:10 2020][8592554.822869] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.829572] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:10 2020][8592554.835612] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.843060] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.850145] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.857897] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:10 2020][8592554.865151] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.873010] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.879977] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:10 2020][8592554.885413] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.891888] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592554.899460] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:10 2020][8592554.904519] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.910802] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:10 2020][8592554.917416] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592554.923683] ll_ost_io00_057 D ffff9c714edd9040 0 87212 2 0x00000080 [Tue Mar 17 18:01:10 2020][8592554.930951] Call Trace: [Tue Mar 17 18:01:10 2020][8592554.933581] [] schedule+0x29/0x70 [Tue Mar 17 18:01:10 2020][8592554.938733] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.945782] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:10 2020][8592554.951793] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.959018] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:10 2020][8592554.964860] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.971591] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.978901] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:10 2020][8592554.985089] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592554.991887] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592554.999203] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.006777] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.013916] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:10 2020][8592555.020710] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:10 2020][8592555.026928] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.033659] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.040305] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.047389] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:10 2020][8592555.054699] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:10 2020][8592555.060452] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:10 2020][8592555.067004] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.074428] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.081509] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.089257] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:10 2020][8592555.096510] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.104368] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.111352] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:10 2020][8592555.116781] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.123254] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.130827] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:10 2020][8592555.135889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592555.142162] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:10 2020][8592555.148775] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592555.155049] ll_ost_io00_058 D ffff9c6dcf4a9040 0 87558 2 0x00000080 [Tue Mar 17 18:01:10 2020][8592555.162319] Call Trace: [Tue Mar 17 18:01:10 2020][8592555.164946] [] schedule+0x29/0x70 [Tue Mar 17 18:01:10 2020][8592555.170090] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.177163] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:10 2020][8592555.183175] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.190402] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:10 2020][8592555.196333] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.203046] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.210352] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:10 2020][8592555.216535] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.223331] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.230648] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.238238] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.245379] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:10 2020][8592555.252174] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:10 2020][8592555.258389] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.265090] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:10 2020][8592555.271037] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.278469] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.285549] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.293298] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:10 2020][8592555.300550] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.308427] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.315392] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:10 2020][8592555.320821] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.327294] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:10 2020][8592555.334867] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:10 2020][8592555.339919] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592555.346187] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:10 2020][8592555.352806] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:10 2020][8592555.359075] ll_ost_io00_059 D ffff9c6dcf4ab0c0 0 87559 2 0x00000080 [Tue Mar 17 18:01:10 2020][8592555.366350] Call Trace: [Tue Mar 17 18:01:10 2020][8592555.368988] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.376650] [] schedule+0x29/0x70 [Tue Mar 17 18:01:10 2020][8592555.381817] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.388886] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:10 2020][8592555.394897] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.402141] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:10 2020][8592555.408854] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:10 2020][8592555.416594] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:10 2020][8592555.422775] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592555.429574] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.436891] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.444477] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.451621] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:11 2020][8592555.457896] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:11 2020][8592555.464689] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:11 2020][8592555.470906] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.477636] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.484286] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.491354] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:11 2020][8592555.498667] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:11 2020][8592555.504431] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:11 2020][8592555.510985] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.518418] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.525496] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.533244] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:11 2020][8592555.540497] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.548357] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.555322] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:11 2020][8592555.560751] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.567226] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.574798] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:11 2020][8592555.579873] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592555.586142] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:11 2020][8592555.592764] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592555.599037] ll_ost_io00_060 D ffff9c6ae9b20000 0 87560 2 0x00000080 [Tue Mar 17 18:01:11 2020][8592555.606305] Call Trace: [Tue Mar 17 18:01:11 2020][8592555.608935] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:11 2020][8592555.615547] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:11 2020][8592555.621900] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:11 2020][8592555.627213] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:11 2020][8592555.632721] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.639691] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.647086] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:11 2020][8592555.653097] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:11 2020][8592555.659457] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:11 2020][8592555.666076] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:11 2020][8592555.671568] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:11 2020][8592555.676797] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:11 2020][8592555.683936] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:11 2020][8592555.689164] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:11 2020][8592555.694851] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.702254] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592555.709222] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.716538] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.723421] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.730733] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:11 2020][8592555.737615] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:11 2020][8592555.743833] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.750572] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.757218] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.764286] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:11 2020][8592555.769803] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.776885] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.784632] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:11 2020][8592555.791883] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.799745] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.806709] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:11 2020][8592555.812138] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.818613] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592555.826183] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:11 2020][8592555.831236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592555.837517] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:11 2020][8592555.844133] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592555.850398] ll_ost_io00_061 D ffff9c6ae9b230c0 0 87561 2 0x00000080 [Tue Mar 17 18:01:11 2020][8592555.857667] Call Trace: [Tue Mar 17 18:01:11 2020][8592555.860297] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:11 2020][8592555.866917] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:11 2020][8592555.873271] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:11 2020][8592555.878593] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:11 2020][8592555.884095] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.891076] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.898475] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:11 2020][8592555.904493] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:11 2020][8592555.910854] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:11 2020][8592555.917470] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:11 2020][8592555.922956] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:11 2020][8592555.928183] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:11 2020][8592555.935323] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:11 2020][8592555.940550] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:11 2020][8592555.946213] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.953628] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592555.960593] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.967906] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.974789] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592555.982102] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:11 2020][8592555.988984] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:11 2020][8592555.995201] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.001903] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:11 2020][8592556.007429] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.014509] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.022270] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:11 2020][8592556.029528] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.037386] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.044353] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:11 2020][8592556.049788] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.056262] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.063835] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:11 2020][8592556.068896] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592556.075170] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:11 2020][8592556.081783] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592556.088063] ll_ost_io00_062 D ffff9c6ae9b25140 0 87562 2 0x00000080 [Tue Mar 17 18:01:11 2020][8592556.095334] Call Trace: [Tue Mar 17 18:01:11 2020][8592556.097963] [] schedule+0x29/0x70 [Tue Mar 17 18:01:11 2020][8592556.103106] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.110157] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:11 2020][8592556.116167] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.123392] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:11 2020][8592556.129235] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.135950] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.143257] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:11 2020][8592556.149462] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.156261] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.163579] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.171154] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.178298] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:11 2020][8592556.185094] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:11 2020][8592556.191311] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.198014] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:11 2020][8592556.203539] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.210620] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.218382] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:11 2020][8592556.225636] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.233499] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.240463] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:11 2020][8592556.245899] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.252373] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.259945] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:11 2020][8592556.265007] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592556.271281] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:11 2020][8592556.277892] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:11 2020][8592556.284174] ll_ost_io00_063 D ffff9c6ae9b24100 0 87563 2 0x00000080 [Tue Mar 17 18:01:11 2020][8592556.291447] Call Trace: [Tue Mar 17 18:01:11 2020][8592556.294074] [] schedule+0x29/0x70 [Tue Mar 17 18:01:11 2020][8592556.299216] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.306268] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:11 2020][8592556.312276] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.319502] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:11 2020][8592556.325337] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.332053] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.339367] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:11 2020][8592556.345572] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:11 2020][8592556.352365] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.359681] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.367253] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:11 2020][8592556.374396] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:11 2020][8592556.381189] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:11 2020][8592556.387418] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.394123] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:11 2020][8592556.400157] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.407587] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.414684] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:11 2020][8592556.422434] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:11 2020][8592556.429684] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.437545] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.444513] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:12 2020][8592556.449949] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.456421] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.463995] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:12 2020][8592556.469058] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592556.475330] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:12 2020][8592556.481960] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592556.488227] ll_ost_io00_064 D ffff9c714edde180 0 87564 2 0x00000080 [Tue Mar 17 18:01:12 2020][8592556.495504] Call Trace: [Tue Mar 17 18:01:12 2020][8592556.498132] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:12 2020][8592556.504745] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:12 2020][8592556.511099] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:12 2020][8592556.516421] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:12 2020][8592556.521920] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.528890] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.536283] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:12 2020][8592556.542292] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:12 2020][8592556.548671] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:12 2020][8592556.555292] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:12 2020][8592556.560784] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:12 2020][8592556.566019] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:12 2020][8592556.573161] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:12 2020][8592556.578396] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:12 2020][8592556.584069] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.591468] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:12 2020][8592556.598438] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.605754] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.612648] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.619967] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:12 2020][8592556.626846] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:12 2020][8592556.633062] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.639765] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:12 2020][8592556.645292] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.652370] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.660119] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:12 2020][8592556.667371] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.675245] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.682207] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:12 2020][8592556.687643] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.694115] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.701688] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:12 2020][8592556.706741] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592556.713016] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:12 2020][8592556.719638] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592556.725912] ll_ost_io02_068 D ffff9c569e9ca080 0 90837 2 0x00000080 [Tue Mar 17 18:01:12 2020][8592556.733189] Call Trace: [Tue Mar 17 18:01:12 2020][8592556.735818] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:12 2020][8592556.741140] [] schedule+0x29/0x70 [Tue Mar 17 18:01:12 2020][8592556.746301] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:12 2020][8592556.752312] [] ? __radix_tree_lookup+0x84/0xf0 [Tue Mar 17 18:01:12 2020][8592556.758589] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:12 2020][8592556.764426] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:12 2020][8592556.769737] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:12 2020][8592556.775917] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:12 2020][8592556.781317] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:12 2020][8592556.786717] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:12 2020][8592556.792738] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:12 2020][8592556.798053] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:12 2020][8592556.804925] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592556.811021] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:12 2020][8592556.816606] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:12 2020][8592556.823489] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.830279] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:12 2020][8592556.837859] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.846125] [] ldiskfs_quota_write+0xc6/0x1e0 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.853175] [] qtree_write_dquot+0xd5/0x180 [Tue Mar 17 18:01:12 2020][8592556.859203] [] v2_write_dquot+0x2b/0x30 [Tue Mar 17 18:01:12 2020][8592556.864865] [] dquot_commit+0xbd/0xd0 [Tue Mar 17 18:01:12 2020][8592556.870357] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.877326] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.884719] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:12 2020][8592556.890993] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.898223] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.905877] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:12 2020][8592556.911463] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.919123] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:01:12 2020][8592556.925677] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.933077] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.940138] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.948324] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:12 2020][8592556.955294] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.962527] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592556.969842] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:12 2020][8592556.976635] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:12 2020][8592556.982854] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.989584] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592556.996231] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.003317] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:12 2020][8592557.010630] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:12 2020][8592557.016377] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:12 2020][8592557.022928] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.030354] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.037435] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.045181] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:12 2020][8592557.052435] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.060294] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.067275] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:12 2020][8592557.072705] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.079193] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.086758] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:12 2020][8592557.091812] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592557.098079] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:12 2020][8592557.104700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592557.110974] ll_ost_io02_069 D ffff9c569e9cb0c0 0 90838 2 0x00000080 [Tue Mar 17 18:01:12 2020][8592557.118242] Call Trace: [Tue Mar 17 18:01:12 2020][8592557.120879] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.128533] [] schedule+0x29/0x70 [Tue Mar 17 18:01:12 2020][8592557.133699] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:12 2020][8592557.140754] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:12 2020][8592557.146771] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:12 2020][8592557.153998] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:12 2020][8592557.160711] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.168451] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:12 2020][8592557.174635] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:12 2020][8592557.181431] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.188747] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.196336] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.203476] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:12 2020][8592557.209755] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:12 2020][8592557.216548] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:12 2020][8592557.222765] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.229467] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:12 2020][8592557.234992] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.242072] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.249818] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:12 2020][8592557.257074] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.264949] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.271915] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:12 2020][8592557.277344] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.283816] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.291389] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:12 2020][8592557.296442] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592557.302709] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:12 2020][8592557.309329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:12 2020][8592557.315594] ll_ost_io02_070 D ffff9c569e9cc100 0 90839 2 0x00000080 [Tue Mar 17 18:01:12 2020][8592557.322864] Call Trace: [Tue Mar 17 18:01:12 2020][8592557.325510] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:12 2020][8592557.332817] [] schedule+0x29/0x70 [Tue Mar 17 18:01:12 2020][8592557.337981] [] rwsem_down_read_failed+0x10d/0x1a0 [Tue Mar 17 18:01:12 2020][8592557.344510] [] call_rwsem_down_read_failed+0x18/0x30 [Tue Mar 17 18:01:12 2020][8592557.351304] [] down_read+0x20/0x40 [Tue Mar 17 18:01:12 2020][8592557.356575] [] osd_read_lock+0x5c/0xe0 [osd_ldiskfs] [Tue Mar 17 18:01:12 2020][8592557.363372] [] ofd_preprw_write.isra.31+0xd3/0xea0 [ofd] [Tue Mar 17 18:01:12 2020][8592557.370513] [] ofd_preprw+0x422/0x11b0 [ofd] [Tue Mar 17 18:01:12 2020][8592557.376643] [] tgt_brw_write+0xc7c/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.383287] [] ? lustre_msg_buf_v2+0x1e0/0x1e0 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.390381] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.397033] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.404083] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:12 2020][8592557.410103] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:12 2020][8592557.415625] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.422704] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:12 2020][8592557.430453] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:13 2020][8592557.437704] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.445566] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.452529] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:13 2020][8592557.457974] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.464450] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.472022] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:13 2020][8592557.477085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.483358] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:13 2020][8592557.489977] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.496244] ll_ost_io02_071 D ffff9c569e9cd140 0 90840 2 0x00000080 [Tue Mar 17 18:01:13 2020][8592557.503513] Call Trace: [Tue Mar 17 18:01:13 2020][8592557.506143] [] schedule+0x29/0x70 [Tue Mar 17 18:01:13 2020][8592557.511295] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.518367] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.525503] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:13 2020][8592557.531513] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:13 2020][8592557.537702] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:13 2020][8592557.544497] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:13 2020][8592557.550713] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.557445] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.564094] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.571163] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:13 2020][8592557.578474] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:13 2020][8592557.584235] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:13 2020][8592557.589758] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.596835] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.604584] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:13 2020][8592557.611837] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.619697] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.626663] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:13 2020][8592557.632098] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.638571] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.646144] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:13 2020][8592557.651211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.657481] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:13 2020][8592557.664101] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.670368] ll_ost_io02_072 D ffff9c569e9ce180 0 90841 2 0x00000080 [Tue Mar 17 18:01:13 2020][8592557.677638] Call Trace: [Tue Mar 17 18:01:13 2020][8592557.680265] [] schedule+0x29/0x70 [Tue Mar 17 18:01:13 2020][8592557.685409] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:13 2020][8592557.692458] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:13 2020][8592557.698469] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:13 2020][8592557.705694] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:13 2020][8592557.711537] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:13 2020][8592557.718267] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.725575] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:13 2020][8592557.731757] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:13 2020][8592557.738555] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.745872] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.753443] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.760585] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:13 2020][8592557.767377] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:13 2020][8592557.773595] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.780341] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.786992] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.794043] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:13 2020][8592557.799568] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.806650] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.814397] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:13 2020][8592557.821649] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.829510] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.836475] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:13 2020][8592557.841911] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.848399] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592557.855966] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:13 2020][8592557.861027] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.867297] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:13 2020][8592557.873914] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592557.880179] ll_ost_io02_073 D ffff9c5392bf9040 0 90842 2 0x00000080 [Tue Mar 17 18:01:13 2020][8592557.887450] Call Trace: [Tue Mar 17 18:01:13 2020][8592557.890079] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:13 2020][8592557.896699] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:13 2020][8592557.903053] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:13 2020][8592557.908389] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:13 2020][8592557.913884] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.920853] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.928248] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:13 2020][8592557.934522] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:01:13 2020][8592557.939676] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.946903] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.954043] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 18:01:13 2020][8592557.960920] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:13 2020][8592557.966499] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.974181] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.981847] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:01:13 2020][8592557.988382] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592557.995784] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.002839] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.011017] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:13 2020][8592558.017988] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.025216] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.032529] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:13 2020][8592558.039323] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:13 2020][8592558.045556] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.052288] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.058938] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.066006] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:13 2020][8592558.073321] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:13 2020][8592558.079074] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:13 2020][8592558.084593] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.091672] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.099418] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:13 2020][8592558.106673] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.114532] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.121514] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:13 2020][8592558.126944] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.133419] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.140990] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:13 2020][8592558.146050] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592558.152317] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:13 2020][8592558.158939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592558.165213] ll_ost_io02_074 D ffff9c5392bfa080 0 90843 2 0x00000080 [Tue Mar 17 18:01:13 2020][8592558.172481] Call Trace: [Tue Mar 17 18:01:13 2020][8592558.175111] [] schedule+0x29/0x70 [Tue Mar 17 18:01:13 2020][8592558.180260] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:13 2020][8592558.187327] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:13 2020][8592558.193338] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:13 2020][8592558.200567] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:13 2020][8592558.207278] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.215019] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:13 2020][8592558.221202] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:13 2020][8592558.228000] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.235315] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.242904] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:13 2020][8592558.250048] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:13 2020][8592558.256324] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:13 2020][8592558.263115] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:13 2020][8592558.269334] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.276064] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.282711] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.289779] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:13 2020][8592558.297096] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:13 2020][8592558.302850] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:13 2020][8592558.309425] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.316852] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.323932] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.331680] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:13 2020][8592558.338934] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.346791] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.353758] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:13 2020][8592558.359214] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.365687] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:13 2020][8592558.373257] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:13 2020][8592558.378326] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592558.384595] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:13 2020][8592558.391230] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:13 2020][8592558.397498] ll_ost_io03_064 D ffff9c5392bfc100 0 90844 2 0x00000080 [Tue Mar 17 18:01:13 2020][8592558.404767] Call Trace: [Tue Mar 17 18:01:13 2020][8592558.407395] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:13 2020][8592558.412708] [] schedule+0x29/0x70 [Tue Mar 17 18:01:13 2020][8592558.417848] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:13 2020][8592558.423864] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:13 2020][8592558.429271] [] ? __find_get_block+0xbc/0x120 [Tue Mar 17 18:01:14 2020][8592558.435386] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:14 2020][8592558.440790] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:14 2020][8592558.446631] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:14 2020][8592558.451952] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:14 2020][8592558.458141] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:14 2020][8592558.463548] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:14 2020][8592558.468955] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:14 2020][8592558.474960] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:14 2020][8592558.480274] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:14 2020][8592558.487146] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592558.493255] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:14 2020][8592558.498833] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.505716] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.513293] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.521554] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:14 2020][8592558.527053] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.534027] [] osd_punch+0x28c/0x6b0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.540651] [] ofd_object_punch+0x7df/0xd30 [ofd] [Tue Mar 17 18:01:14 2020][8592558.547184] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:14 2020][8592558.553491] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.560592] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.568341] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:14 2020][8592558.575595] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.583454] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.590421] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:14 2020][8592558.595856] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.602330] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.609904] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:14 2020][8592558.614963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592558.621230] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:14 2020][8592558.627865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592558.634136] ll_ost_io02_075 D ffff9c5392bfb0c0 0 90846 2 0x00000080 [Tue Mar 17 18:01:14 2020][8592558.641413] Call Trace: [Tue Mar 17 18:01:14 2020][8592558.644040] [] schedule+0x29/0x70 [Tue Mar 17 18:01:14 2020][8592558.649185] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.656234] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:14 2020][8592558.662243] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.669473] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.676186] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.683500] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:14 2020][8592558.689682] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.696494] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.703813] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.711388] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.718526] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:14 2020][8592558.725321] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:14 2020][8592558.731538] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.738240] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:14 2020][8592558.743907] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:14 2020][8592558.749662] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:14 2020][8592558.756229] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.763655] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.770738] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.778484] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:14 2020][8592558.785735] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.793598] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.800564] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:14 2020][8592558.806345] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.812819] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.820391] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:14 2020][8592558.825452] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592558.831733] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:14 2020][8592558.838349] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592558.844624] ll_ost_io03_065 D ffff9c5a36f7d140 0 90847 2 0x00000080 [Tue Mar 17 18:01:14 2020][8592558.851899] Call Trace: [Tue Mar 17 18:01:14 2020][8592558.854528] [] schedule+0x29/0x70 [Tue Mar 17 18:01:14 2020][8592558.859673] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.866723] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:14 2020][8592558.872733] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.879958] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:14 2020][8592558.885802] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.892516] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.899832] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:14 2020][8592558.906021] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592558.912819] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.920134] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.927709] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592558.934848] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:14 2020][8592558.941644] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:14 2020][8592558.947860] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.954578] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:14 2020][8592558.960105] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.967186] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.974931] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:14 2020][8592558.982187] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.990043] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592558.997011] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:14 2020][8592559.002448] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.008921] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.016492] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:14 2020][8592559.021546] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592559.027829] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:14 2020][8592559.034441] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592559.040706] ll_ost_io02_076 D ffff9c5a1dbd2080 0 90848 2 0x00000080 [Tue Mar 17 18:01:14 2020][8592559.047976] Call Trace: [Tue Mar 17 18:01:14 2020][8592559.050607] [] schedule+0x29/0x70 [Tue Mar 17 18:01:14 2020][8592559.055755] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.062807] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:14 2020][8592559.068818] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.076042] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:14 2020][8592559.081895] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.088610] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.095923] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:14 2020][8592559.102106] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.108902] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.116220] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.123793] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.130932] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:14 2020][8592559.137727] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:14 2020][8592559.143945] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.150660] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:14 2020][8592559.156185] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.163268] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.171016] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:14 2020][8592559.178268] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.186120] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 18:01:14 2020][8592559.193462] [] ? ptlrpc_at_add_timed+0xe5/0x230 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.200631] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.207103] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.214674] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:14 2020][8592559.219728] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592559.226017] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:14 2020][8592559.232633] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592559.238906] ll_ost_io02_077 D ffff9c5a1dbd1040 0 90849 2 0x00000080 [Tue Mar 17 18:01:14 2020][8592559.246176] Call Trace: [Tue Mar 17 18:01:14 2020][8592559.248805] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:14 2020][8592559.254126] [] schedule+0x29/0x70 [Tue Mar 17 18:01:14 2020][8592559.259275] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:14 2020][8592559.265290] [] ? __radix_tree_lookup+0x84/0xf0 [Tue Mar 17 18:01:14 2020][8592559.271563] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:14 2020][8592559.276892] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:14 2020][8592559.283073] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:14 2020][8592559.288473] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:14 2020][8592559.293881] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:14 2020][8592559.299895] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:14 2020][8592559.305208] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:14 2020][8592559.312089] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:14 2020][8592559.318182] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:14 2020][8592559.323759] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.330642] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.337434] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:14 2020][8592559.345029] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.353299] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.361154] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.367795] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:01:14 2020][8592559.374621] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.382576] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.389749] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:01:14 2020][8592559.396487] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:01:14 2020][8592559.403285] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.410343] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:14 2020][8592559.417509] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:14 2020][8592559.424820] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:14 2020][8592559.431008] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:15 2020][8592559.437806] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:15 2020][8592559.444021] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.450751] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.457401] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.464450] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:15 2020][8592559.469968] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.477065] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.484813] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:15 2020][8592559.492066] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.499924] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.506891] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:15 2020][8592559.512319] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.518792] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.526365] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:15 2020][8592559.531418] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592559.537693] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:15 2020][8592559.544329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592559.550598] ll_ost_io02_078 D ffff9c5a1dbd0000 0 90850 2 0x00000080 [Tue Mar 17 18:01:15 2020][8592559.557883] Call Trace: [Tue Mar 17 18:01:15 2020][8592559.560514] [] schedule+0x29/0x70 [Tue Mar 17 18:01:15 2020][8592559.565663] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:01:15 2020][8592559.573151] [] ? mempool_alloc+0x6e/0x170 [Tue Mar 17 18:01:15 2020][8592559.578991] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:15 2020][8592559.585007] [] raid5_make_request+0x1a5/0xca0 [raid456] [Tue Mar 17 18:01:15 2020][8592559.592058] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:01:15 2020][8592559.597199] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:15 2020][8592559.603234] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.611069] [] md_handle_request+0xd0/0x150 [Tue Mar 17 18:01:15 2020][8592559.617083] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 18:01:15 2020][8592559.624222] [] md_make_request+0x79/0x190 [Tue Mar 17 18:01:15 2020][8592559.630063] [] generic_make_request+0x147/0x380 [Tue Mar 17 18:01:15 2020][8592559.636425] [] ? md_mergeable_bvec+0x46/0x50 [Tue Mar 17 18:01:15 2020][8592559.642525] [] submit_bio+0x70/0x150 [Tue Mar 17 18:01:15 2020][8592559.647941] [] ? lprocfs_oh_tally+0x17/0x40 [obdclass] [Tue Mar 17 18:01:15 2020][8592559.654909] [] osd_submit_bio+0x1c/0x60 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.661789] [] osd_do_bio.isra.35+0x9b5/0xab0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.669200] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:15 2020][8592559.675385] [] ? jbd2_journal_add_journal_head+0x65/0x150 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.683216] [] ? recalc_bh_state.part.21+0x70/0x90 [Tue Mar 17 18:01:15 2020][8592559.689840] [] ? jbd2_journal_get_write_access+0x32/0x40 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.697592] [] osd_write_commit+0x3ec/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.704818] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.712132] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:15 2020][8592559.718928] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:15 2020][8592559.725143] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.731845] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:15 2020][8592559.737528] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:15 2020][8592559.743277] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:15 2020][8592559.748802] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.755883] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.763631] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:15 2020][8592559.770883] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.778745] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.785709] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:15 2020][8592559.791147] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.797621] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.805206] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:15 2020][8592559.810260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592559.816528] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:15 2020][8592559.823149] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592559.829416] ll_ost_io02_079 D ffff9c5a1dbd30c0 0 90851 2 0x00000080 [Tue Mar 17 18:01:15 2020][8592559.836692] Call Trace: [Tue Mar 17 18:01:15 2020][8592559.839330] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.846989] [] schedule+0x29/0x70 [Tue Mar 17 18:01:15 2020][8592559.852132] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.859185] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:15 2020][8592559.865194] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.872437] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.879151] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.886894] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:15 2020][8592559.893082] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592559.899882] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.907196] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.914771] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592559.921910] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:15 2020][8592559.928185] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:15 2020][8592559.934993] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:15 2020][8592559.941204] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.947936] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.954585] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.961652] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:15 2020][8592559.968959] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:15 2020][8592559.974712] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:15 2020][8592559.980232] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.987310] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592559.995058] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:15 2020][8592560.002326] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.010189] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.017153] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:15 2020][8592560.022581] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.029056] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.036629] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:15 2020][8592560.041689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592560.047954] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:15 2020][8592560.054569] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592560.060843] ll_ost_io03_066 D ffff9c5a36a9a080 0 90852 2 0x00000080 [Tue Mar 17 18:01:15 2020][8592560.068137] Call Trace: [Tue Mar 17 18:01:15 2020][8592560.070775] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.078437] [] schedule+0x29/0x70 [Tue Mar 17 18:01:15 2020][8592560.083587] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.090639] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:15 2020][8592560.096657] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.103884] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.110597] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.118338] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:15 2020][8592560.124520] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.131331] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.138651] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.146225] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.153362] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:15 2020][8592560.159638] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:15 2020][8592560.166435] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:15 2020][8592560.172651] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.179382] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.186030] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.193099] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:15 2020][8592560.200426] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:15 2020][8592560.206175] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:15 2020][8592560.212738] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.220170] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.227257] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.235007] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:15 2020][8592560.242258] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.250118] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.257083] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:15 2020][8592560.262520] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.269010] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.276583] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:15 2020][8592560.281637] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592560.287912] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:15 2020][8592560.294533] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:15 2020][8592560.300808] ll_ost_io03_067 D ffff9c535cc09040 0 90853 2 0x00000080 [Tue Mar 17 18:01:15 2020][8592560.308083] Call Trace: [Tue Mar 17 18:01:15 2020][8592560.310722] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.318383] [] schedule+0x29/0x70 [Tue Mar 17 18:01:15 2020][8592560.323534] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.330599] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:15 2020][8592560.336615] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.343842] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.350549] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.358295] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:15 2020][8592560.364484] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:15 2020][8592560.371283] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.378598] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.386173] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:15 2020][8592560.393327] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:15 2020][8592560.399617] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:15 2020][8592560.406413] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:15 2020][8592560.412633] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.419365] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.426011] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:15 2020][8592560.433078] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:16 2020][8592560.440385] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:16 2020][8592560.446141] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:16 2020][8592560.451667] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.458746] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.466510] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:16 2020][8592560.473763] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.481624] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.488588] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:16 2020][8592560.494015] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.500492] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.508064] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:16 2020][8592560.513123] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.519391] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:16 2020][8592560.526013] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.532301] ll_ost_io00_065 D ffff9c5a3a2b6180 0 90854 2 0x00000080 [Tue Mar 17 18:01:16 2020][8592560.539572] Call Trace: [Tue Mar 17 18:01:16 2020][8592560.542201] [] schedule+0x29/0x70 [Tue Mar 17 18:01:16 2020][8592560.547353] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.554404] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:16 2020][8592560.560413] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.567642] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.574352] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.582095] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:16 2020][8592560.588284] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.595099] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.602415] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.609989] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.617130] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:16 2020][8592560.623404] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:16 2020][8592560.630198] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:16 2020][8592560.636414] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.643147] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.649793] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.656844] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:16 2020][8592560.662890] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.670320] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.677398] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.685144] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:16 2020][8592560.692398] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.700260] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.707224] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:16 2020][8592560.712663] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.719135] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.726708] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:16 2020][8592560.731769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.738051] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:16 2020][8592560.744665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.750939] ll_ost_io02_080 D ffff9c5a3a2b0000 0 90855 2 0x00000080 [Tue Mar 17 18:01:16 2020][8592560.758208] Call Trace: [Tue Mar 17 18:01:16 2020][8592560.760837] [] schedule+0x29/0x70 [Tue Mar 17 18:01:16 2020][8592560.765979] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.773030] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:16 2020][8592560.779040] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.786268] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.792980] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.800721] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:16 2020][8592560.806911] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592560.813709] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.821027] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.828597] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592560.835739] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:16 2020][8592560.842013] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:16 2020][8592560.848807] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:16 2020][8592560.855027] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.861750] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:16 2020][8592560.867701] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.874350] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.881400] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:16 2020][8592560.887145] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:16 2020][8592560.893698] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.901133] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.908212] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.915959] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:16 2020][8592560.923212] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.931088] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.938054] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:16 2020][8592560.943483] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.949955] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592560.957528] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:16 2020][8592560.962589] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.968857] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:16 2020][8592560.975476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592560.981742] ll_ost_io02_081 D ffff9c54f58c2080 0 90856 2 0x00000080 [Tue Mar 17 18:01:16 2020][8592560.989021] Call Trace: [Tue Mar 17 18:01:16 2020][8592560.991650] [] schedule+0x29/0x70 [Tue Mar 17 18:01:16 2020][8592560.996802] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.003866] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:16 2020][8592561.009878] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.017104] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:16 2020][8592561.022947] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.029661] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.036966] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:16 2020][8592561.043151] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.049947] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.057264] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.064838] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.071977] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:16 2020][8592561.078770] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:16 2020][8592561.084986] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.091719] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.098366] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.105434] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:16 2020][8592561.112742] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:16 2020][8592561.118494] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:16 2020][8592561.125046] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.132489] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.139570] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.147318] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:16 2020][8592561.154569] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.162431] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.169395] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:16 2020][8592561.174823] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.181296] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.188870] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:16 2020][8592561.193930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592561.200197] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:16 2020][8592561.206818] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592561.213091] ll_ost_io00_066 D ffff9c54f58c6180 0 90857 2 0x00000080 [Tue Mar 17 18:01:16 2020][8592561.220360] Call Trace: [Tue Mar 17 18:01:16 2020][8592561.222991] [] schedule+0x29/0x70 [Tue Mar 17 18:01:16 2020][8592561.228141] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.235193] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:16 2020][8592561.241203] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.248427] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:16 2020][8592561.254273] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.260998] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.268309] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:16 2020][8592561.274499] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:16 2020][8592561.281298] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.288614] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.296188] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:16 2020][8592561.303325] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:16 2020][8592561.310121] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:16 2020][8592561.316338] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.323053] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:16 2020][8592561.329004] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.335653] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.342703] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:16 2020][8592561.348736] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.356186] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.363277] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.371039] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:16 2020][8592561.378292] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.386152] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.393133] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:16 2020][8592561.398579] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.405054] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:16 2020][8592561.412627] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:16 2020][8592561.417689] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:16 2020][8592561.423964] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:16 2020][8592561.430585] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592561.436860] ll_ost_io03_068 D ffff9c54f58c5140 0 90858 2 0x00000080 [Tue Mar 17 18:01:17 2020][8592561.444137] Call Trace: [Tue Mar 17 18:01:17 2020][8592561.446766] [] schedule+0x29/0x70 [Tue Mar 17 18:01:17 2020][8592561.451919] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.458981] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:17 2020][8592561.464993] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.472219] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:17 2020][8592561.478053] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.484769] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.492086] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:17 2020][8592561.498275] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.505071] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.512387] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.519976] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.527120] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:17 2020][8592561.533913] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:17 2020][8592561.540131] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.546861] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.553509] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.560561] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:17 2020][8592561.566599] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.574036] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.581123] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.588885] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:17 2020][8592561.596139] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.603999] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.610964] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:17 2020][8592561.616393] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.622866] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.630440] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:17 2020][8592561.635500] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592561.641767] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:17 2020][8592561.648387] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592561.654670] ll_ost_io02_082 D ffff9c54f58c30c0 0 90859 2 0x00000080 [Tue Mar 17 18:01:17 2020][8592561.661940] Call Trace: [Tue Mar 17 18:01:17 2020][8592561.664569] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:17 2020][8592561.671190] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:17 2020][8592561.677552] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:17 2020][8592561.682872] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:17 2020][8592561.688368] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.695342] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.702737] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:17 2020][8592561.708760] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:17 2020][8592561.715132] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:17 2020][8592561.721757] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.728463] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:17 2020][8592561.735602] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:17 2020][8592561.740835] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:17 2020][8592561.746502] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.753901] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.760872] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.768186] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.775065] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.782392] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:17 2020][8592561.789268] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:17 2020][8592561.795487] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.802216] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.808866] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.815935] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:17 2020][8592561.823249] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:17 2020][8592561.828774] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.835854] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.843600] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:17 2020][8592561.850857] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.858729] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.865727] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.872203] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592561.879772] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:17 2020][8592561.884835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592561.891109] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:17 2020][8592561.897730] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592561.904005] ll_ost_io03_069 D ffff9c54f58c0000 0 90860 2 0x00000080 [Tue Mar 17 18:01:17 2020][8592561.911282] Call Trace: [Tue Mar 17 18:01:17 2020][8592561.913909] [] schedule+0x29/0x70 [Tue Mar 17 18:01:17 2020][8592561.919054] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.926121] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:17 2020][8592561.932130] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.939358] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.946073] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.953387] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:17 2020][8592561.959571] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592561.966367] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.973684] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.981273] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592561.988414] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:17 2020][8592561.995208] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:17 2020][8592562.001426] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.008127] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:17 2020][8592562.014169] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.021599] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.028680] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.036429] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:17 2020][8592562.043680] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.051541] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.058521] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:17 2020][8592562.063950] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.070427] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.077997] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:17 2020][8592562.083050] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592562.089317] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:17 2020][8592562.095938] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592562.102213] ll_ost_io03_070 D ffff9c56cb4ee180 0 90861 2 0x00000080 [Tue Mar 17 18:01:17 2020][8592562.109489] Call Trace: [Tue Mar 17 18:01:17 2020][8592562.112117] [] schedule+0x29/0x70 [Tue Mar 17 18:01:17 2020][8592562.117262] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:01:17 2020][8592562.124761] [] ? mempool_alloc+0x6e/0x170 [Tue Mar 17 18:01:17 2020][8592562.130595] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:17 2020][8592562.136603] [] raid5_make_request+0x1a5/0xca0 [raid456] [Tue Mar 17 18:01:17 2020][8592562.143657] [] ? dqput+0x4c/0x1f0 [Tue Mar 17 18:01:17 2020][8592562.148796] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:17 2020][8592562.154811] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.162639] [] md_handle_request+0xd0/0x150 [Tue Mar 17 18:01:17 2020][8592562.168652] [] ? generic_make_request_checks+0x2a7/0x440 [Tue Mar 17 18:01:17 2020][8592562.175793] [] md_make_request+0x79/0x190 [Tue Mar 17 18:01:17 2020][8592562.181633] [] generic_make_request+0x147/0x380 [Tue Mar 17 18:01:17 2020][8592562.187987] [] ? md_mergeable_bvec+0x46/0x50 [Tue Mar 17 18:01:17 2020][8592562.194087] [] submit_bio+0x70/0x150 [Tue Mar 17 18:01:17 2020][8592562.199505] [] ? lprocfs_oh_tally+0x17/0x40 [obdclass] [Tue Mar 17 18:01:17 2020][8592562.206481] [] osd_submit_bio+0x1c/0x60 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.213362] [] osd_do_bio.isra.35+0x9b5/0xab0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.220758] [] ? recalc_bh_state.part.21+0x70/0x90 [Tue Mar 17 18:01:17 2020][8592562.227381] [] ? jbd2_journal_get_write_access+0x32/0x40 [jbd2] [Tue Mar 17 18:01:17 2020][8592562.235132] [] osd_write_commit+0x3ec/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.242361] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.249689] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:17 2020][8592562.256484] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:17 2020][8592562.262701] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.269431] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.276079] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.283148] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:17 2020][8592562.290464] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:17 2020][8592562.296216] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:17 2020][8592562.301735] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.308818] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.316577] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:17 2020][8592562.323833] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.331694] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.338660] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:17 2020][8592562.344095] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.350574] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:17 2020][8592562.358196] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:17 2020][8592562.363254] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592562.369521] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:17 2020][8592562.376154] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:17 2020][8592562.382425] ll_ost_io02_083 D ffff9c56cb4ea080 0 90862 2 0x00000080 [Tue Mar 17 18:01:17 2020][8592562.389694] Call Trace: [Tue Mar 17 18:01:17 2020][8592562.392330] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:17 2020][8592562.400000] [] schedule+0x29/0x70 [Tue Mar 17 18:01:17 2020][8592562.405143] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:17 2020][8592562.412197] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:17 2020][8592562.418213] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:17 2020][8592562.425441] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:17 2020][8592562.432151] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.439893] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:18 2020][8592562.446090] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592562.452885] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.460200] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.467780] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.474921] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:18 2020][8592562.481196] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:18 2020][8592562.487989] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:18 2020][8592562.494214] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.500920] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:18 2020][8592562.506601] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:18 2020][8592562.512349] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:18 2020][8592562.517875] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.524954] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.532704] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:18 2020][8592562.539956] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.547815] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.554781] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:18 2020][8592562.560219] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.566690] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.574262] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:18 2020][8592562.579331] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.585598] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:18 2020][8592562.592212] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.598488] ll_ost_io00_067 D ffff9c56cb4ed140 0 90863 2 0x00000080 [Tue Mar 17 18:01:18 2020][8592562.605764] Call Trace: [Tue Mar 17 18:01:18 2020][8592562.608394] [] schedule+0x29/0x70 [Tue Mar 17 18:01:18 2020][8592562.613544] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:18 2020][8592562.620595] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:18 2020][8592562.626613] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592562.633856] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:18 2020][8592562.639692] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:18 2020][8592562.646406] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.653719] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:18 2020][8592562.659903] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592562.666699] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.674016] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.681591] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.688729] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:18 2020][8592562.695524] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:18 2020][8592562.701755] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.708460] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:18 2020][8592562.713983] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.721064] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.728811] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:18 2020][8592562.736066] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.743926] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.750892] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:18 2020][8592562.756328] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.762802] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.770389] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:18 2020][8592562.775442] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.781709] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:18 2020][8592562.788321] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.794589] ll_ost_io02_084 D ffff9c56cb4ec100 0 90864 2 0x00000080 [Tue Mar 17 18:01:18 2020][8592562.801866] Call Trace: [Tue Mar 17 18:01:18 2020][8592562.804494] [] schedule+0x29/0x70 [Tue Mar 17 18:01:18 2020][8592562.809641] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.816694] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.823829] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:18 2020][8592562.829856] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:18 2020][8592562.836047] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:18 2020][8592562.842840] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:18 2020][8592562.849057] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.855759] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:18 2020][8592562.861285] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.868366] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.876111] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:18 2020][8592562.883367] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.891226] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.898206] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:18 2020][8592562.903636] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.910108] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592562.917684] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:18 2020][8592562.922744] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.929019] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:18 2020][8592562.935640] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592562.941915] ll_ost_io00_068 D ffff9c5a17bd4100 0 90865 2 0x00000080 [Tue Mar 17 18:01:18 2020][8592562.949192] Call Trace: [Tue Mar 17 18:01:18 2020][8592562.951821] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:18 2020][8592562.958443] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:18 2020][8592562.964817] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:18 2020][8592562.970134] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:18 2020][8592562.975635] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.982604] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592562.989998] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:18 2020][8592562.996274] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:18 2020][8592563.002642] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.009867] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.017010] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 18:01:18 2020][8592563.023899] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:18 2020][8592563.029484] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.037154] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.044820] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:01:18 2020][8592563.051356] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.058756] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.065815] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.074000] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592563.080972] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.088197] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.095526] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:18 2020][8592563.102321] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:18 2020][8592563.108541] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.115248] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:18 2020][8592563.121282] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.128713] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.135795] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.143541] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:18 2020][8592563.150794] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.158656] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.165636] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:18 2020][8592563.171066] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.177540] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.185111] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:18 2020][8592563.190164] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592563.196431] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:18 2020][8592563.203052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592563.209324] ll_ost_io03_071 D ffff9c5a29768000 0 90866 2 0x00000080 [Tue Mar 17 18:01:18 2020][8592563.216595] Call Trace: [Tue Mar 17 18:01:18 2020][8592563.219223] [] schedule+0x29/0x70 [Tue Mar 17 18:01:18 2020][8592563.224366] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:18 2020][8592563.231432] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:18 2020][8592563.237444] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592563.244669] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:18 2020][8592563.250515] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:18 2020][8592563.257228] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.264542] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:18 2020][8592563.270734] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:18 2020][8592563.277530] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.284847] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.292436] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:18 2020][8592563.299576] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:18 2020][8592563.306371] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:18 2020][8592563.312590] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.319329] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.325978] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.333027] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:18 2020][8592563.339069] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.346501] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.353582] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.361348] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:18 2020][8592563.368613] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.376478] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.383440] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:18 2020][8592563.388870] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.395343] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:18 2020][8592563.402932] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:18 2020][8592563.407984] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592563.414264] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:18 2020][8592563.420881] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:18 2020][8592563.427156] ll_ost_io02_085 D ffff9c56cc3c0000 0 90867 2 0x00000080 [Tue Mar 17 18:01:18 2020][8592563.434441] Call Trace: [Tue Mar 17 18:01:18 2020][8592563.437072] [] schedule+0x29/0x70 [Tue Mar 17 18:01:19 2020][8592563.442221] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.449275] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:19 2020][8592563.455290] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.462519] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:19 2020][8592563.468361] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.475074] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.482389] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:19 2020][8592563.488595] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.495395] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.502710] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.510284] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.517425] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:19 2020][8592563.524219] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:19 2020][8592563.530437] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.537176] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.543821] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.550895] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:19 2020][8592563.558207] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:19 2020][8592563.563974] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:19 2020][8592563.569498] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.576583] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.584330] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:19 2020][8592563.591584] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.599445] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.606410] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:19 2020][8592563.611846] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.618321] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.625907] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:19 2020][8592563.630963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592563.637238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:19 2020][8592563.643858] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592563.650125] ll_ost_io00_069 D ffff9c56cc3c1040 0 90868 2 0x00000080 [Tue Mar 17 18:01:19 2020][8592563.657401] Call Trace: [Tue Mar 17 18:01:19 2020][8592563.660038] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.667691] [] schedule+0x29/0x70 [Tue Mar 17 18:01:19 2020][8592563.672836] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.679887] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:19 2020][8592563.685904] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.693148] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.699861] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.707604] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:19 2020][8592563.713792] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.720593] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.727907] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.735480] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.742622] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:19 2020][8592563.748895] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:19 2020][8592563.755703] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:19 2020][8592563.761925] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.768655] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.775302] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.782353] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:19 2020][8592563.787871] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.794950] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.802698] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:19 2020][8592563.809951] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.817812] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.824792] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:19 2020][8592563.830223] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.836697] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592563.844267] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:19 2020][8592563.849320] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592563.855587] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:19 2020][8592563.862200] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592563.868474] ll_ost_io03_072 D ffff9c56cc3c2080 0 90869 2 0x00000080 [Tue Mar 17 18:01:19 2020][8592563.875742] Call Trace: [Tue Mar 17 18:01:19 2020][8592563.878371] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:19 2020][8592563.884985] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:19 2020][8592563.891361] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:19 2020][8592563.896677] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:19 2020][8592563.902180] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.909152] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.916549] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:19 2020][8592563.922570] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:19 2020][8592563.928928] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:19 2020][8592563.935551] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.942255] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:19 2020][8592563.949411] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:19 2020][8592563.954640] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:19 2020][8592563.960313] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.967711] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592563.974682] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.981999] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.988878] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592563.996192] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:19 2020][8592564.003071] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:19 2020][8592564.009289] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.016005] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:19 2020][8592564.021528] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.028607] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.036354] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:19 2020][8592564.043606] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.051466] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.058431] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:19 2020][8592564.063859] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.070333] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.077907] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:19 2020][8592564.082982] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592564.089252] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:19 2020][8592564.095873] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592564.102146] ll_ost_io02_086 D ffff9c56cc3c30c0 0 90870 2 0x00000080 [Tue Mar 17 18:01:19 2020][8592564.109416] Call Trace: [Tue Mar 17 18:01:19 2020][8592564.112045] [] schedule+0x29/0x70 [Tue Mar 17 18:01:19 2020][8592564.117197] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.124253] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.131387] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:19 2020][8592564.137399] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:19 2020][8592564.143600] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:19 2020][8592564.150397] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:19 2020][8592564.156615] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.163316] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:19 2020][8592564.169274] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.175921] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.182975] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:19 2020][8592564.189013] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.196448] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.203527] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.211288] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:19 2020][8592564.218543] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.226406] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.233369] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:19 2020][8592564.238808] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.245280] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:19 2020][8592564.252852] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:19 2020][8592564.257905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592564.264171] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:19 2020][8592564.270783] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:19 2020][8592564.277064] ll_ost_io00_070 D ffff9c56cc3c4100 0 90871 2 0x00000080 [Tue Mar 17 18:01:19 2020][8592564.284335] Call Trace: [Tue Mar 17 18:01:19 2020][8592564.286964] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:19 2020][8592564.293578] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:19 2020][8592564.299939] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:19 2020][8592564.305261] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:19 2020][8592564.310762] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.317730] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.325125] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:19 2020][8592564.331144] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:19 2020][8592564.337519] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:19 2020][8592564.344145] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.350848] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:19 2020][8592564.358008] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:19 2020][8592564.363239] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:19 2020][8592564.368908] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.376304] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:19 2020][8592564.383276] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.390590] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.397471] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:19 2020][8592564.404813] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:19 2020][8592564.411690] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.417908] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.424638] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.431286] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.438358] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:20 2020][8592564.445671] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:20 2020][8592564.451426] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:20 2020][8592564.457984] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.460194] LNet: Service thread pid 68767 was inactive for 200.23s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [Tue Mar 17 18:01:20 2020][8592564.460197] Pid: 68767, comm: ll_ost01_070 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 SMP Thu Nov 7 15:26:16 PST 2019 [Tue Mar 17 18:01:20 2020][8592564.460197] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.460236] [] jbd2_log_wait_commit+0xc5/0x140 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.460241] [] jbd2_complete_transaction+0x52/0xa0 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.460249] [] ldiskfs_sync_file+0x292/0x320 [ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.460254] [] vfs_fsync_range+0x20/0x30 [Tue Mar 17 18:01:20 2020][8592564.460268] [] osd_object_sync+0xb1/0x160 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.460329] [] tgt_sync+0xb7/0x270 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.460335] [] ofd_sync_hdl+0x111/0x530 [ofd] [Tue Mar 17 18:01:20 2020][8592564.460372] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.460405] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.460437] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.460442] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.460446] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.460459] [] 0xffffffffffffffff [Tue Mar 17 18:01:20 2020][8592564.460461] LustreError: dumping log to /tmp/lustre-log.1584493280.68767 [Tue Mar 17 18:01:20 2020][8592564.586780] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.593859] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.601608] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:20 2020][8592564.608859] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.616720] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.623699] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:20 2020][8592564.629129] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.635604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.643176] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.648237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592564.654503] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.661115] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592564.667382] ll_ost_io03_073 D ffff9c56cc3c5140 0 90872 2 0x00000080 [Tue Mar 17 18:01:20 2020][8592564.674658] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.677287] [] schedule+0x29/0x70 [Tue Mar 17 18:01:20 2020][8592564.682433] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.689496] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:20 2020][8592564.695507] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.702733] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:20 2020][8592564.708568] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.715286] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.722599] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:20 2020][8592564.728787] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:20 2020][8592564.735585] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.742903] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.750490] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.757634] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:20 2020][8592564.764429] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.770645] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.777374] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.784022] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.791074] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:20 2020][8592564.796600] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.803680] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.811442] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:20 2020][8592564.818699] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.826559] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.833522] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:20 2020][8592564.838953] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.845426] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.852998] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.858059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592564.864332] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.870947] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592564.877235] ll_ost_io03_074 D ffff9c56cc3c6180 0 90873 2 0x00000080 [Tue Mar 17 18:01:20 2020][8592564.884508] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.887136] [] schedule+0x29/0x70 [Tue Mar 17 18:01:20 2020][8592564.892291] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.899344] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.906479] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:20 2020][8592564.912488] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:20 2020][8592564.918676] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:20 2020][8592564.925472] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.931690] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.938407] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:20 2020][8592564.944358] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.951005] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.958057] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:20 2020][8592564.963810] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:20 2020][8592564.969336] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972197] LNet: Service thread pid 66952 was inactive for 200.36s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: [Tue Mar 17 18:01:20 2020][8592564.972199] Pid: 66952, comm: ll_ost_io01_000 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 SMP Thu Nov 7 15:26:16 PST 2019 [Tue Mar 17 18:01:20 2020][8592564.972199] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.972224] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.972232] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972237] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972243] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972299] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972335] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972368] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972400] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972406] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.972410] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.972435] [] 0xffffffffffffffff [Tue Mar 17 18:01:20 2020][8592564.972439] Pid: 68884, comm: ll_ost_io01_050 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 SMP Thu Nov 7 15:26:16 PST 2019 [Tue Mar 17 18:01:20 2020][8592564.972439] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.972449] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.972454] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972459] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972464] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972499] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972533] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972564] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972595] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972596] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.972598] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.972601] [] 0xffffffffffffffff [Tue Mar 17 18:01:20 2020][8592564.972603] Pid: 68852, comm: ll_ost_io02_043 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 SMP Thu Nov 7 15:26:16 PST 2019 [Tue Mar 17 18:01:20 2020][8592564.972604] Call Trace: [Tue Mar 17 18:01:20 2020][8592564.972614] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592564.972618] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972623] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972628] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:20 2020][8592564.972662] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972695] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972726] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972756] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592564.972757] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592564.972759] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592564.972762] [] 0xffffffffffffffff [Tue Mar 17 18:01:20 2020][8592565.247001] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592565.254748] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:20 2020][8592565.262000] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592565.269861] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592565.276843] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:20 2020][8592565.282274] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592565.288747] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:20 2020][8592565.296317] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:20 2020][8592565.301370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592565.307637] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:20 2020][8592565.314259] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:20 2020][8592565.320531] ll_ost_io02_087 D ffff9c4cb1448000 0 90874 2 0x00000080 [Tue Mar 17 18:01:20 2020][8592565.327800] Call Trace: [Tue Mar 17 18:01:20 2020][8592565.330439] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:20 2020][8592565.338100] [] schedule+0x29/0x70 [Tue Mar 17 18:01:20 2020][8592565.343266] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:20 2020][8592565.350319] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:20 2020][8592565.356331] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:20 2020][8592565.363556] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:20 2020][8592565.370267] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592565.378011] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:20 2020][8592565.384200] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:20 2020][8592565.391000] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592565.398314] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:20 2020][8592565.405905] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:20 2020][8592565.413061] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.419338] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:21 2020][8592565.426132] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.432348] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.439094] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.445747] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.452818] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:21 2020][8592565.460129] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:21 2020][8592565.465883] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:21 2020][8592565.471425] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.478506] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.484203] Pid: 68506, comm: ll_ost_io02_019 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 SMP Thu Nov 7 15:26:16 PST 2019 [Tue Mar 17 18:01:21 2020][8592565.484203] Call Trace: [Tue Mar 17 18:01:21 2020][8592565.484227] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.484235] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:21 2020][8592565.484241] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:21 2020][8592565.484247] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.484293] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.484329] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.484361] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.484393] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.484398] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:21 2020][8592565.484401] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:21 2020][8592565.484427] [] 0xffffffffffffffff [Tue Mar 17 18:01:21 2020][8592565.484429] LustreError: dumping log to /tmp/lustre-log.1584493281.68506 [Tue Mar 17 18:01:21 2020][8592565.485076] LNet: Service thread pid 40808 was inactive for 200.57s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:21 2020][8592565.590670] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:21 2020][8592565.597923] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.605783] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.612763] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:21 2020][8592565.618197] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.624676] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.632249] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:21 2020][8592565.637310] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592565.643575] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:21 2020][8592565.650191] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592565.656462] ll_ost_io02_088 D ffff9c4cb1449040 0 90875 2 0x00000080 [Tue Mar 17 18:01:21 2020][8592565.663731] Call Trace: [Tue Mar 17 18:01:21 2020][8592565.666360] [] schedule+0x29/0x70 [Tue Mar 17 18:01:21 2020][8592565.671504] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.678572] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:21 2020][8592565.684582] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.691808] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.698523] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.705829] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:21 2020][8592565.712010] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.718809] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.726127] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.733707] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.740862] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:21 2020][8592565.747657] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.753877] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.760607] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.767255] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.774307] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:21 2020][8592565.780344] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.787771] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.794850] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.802597] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:21 2020][8592565.809851] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.817727] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.824694] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:21 2020][8592565.830130] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.836604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.844176] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:21 2020][8592565.849237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592565.855505] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:21 2020][8592565.862125] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592565.868401] ll_ost_io02_089 D ffff9c4cb144a080 0 90876 2 0x00000080 [Tue Mar 17 18:01:21 2020][8592565.875695] Call Trace: [Tue Mar 17 18:01:21 2020][8592565.878330] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.885984] [] schedule+0x29/0x70 [Tue Mar 17 18:01:21 2020][8592565.891128] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.898179] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:21 2020][8592565.904197] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.911425] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.918135] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.925878] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:21 2020][8592565.932069] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:21 2020][8592565.938882] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.946201] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.953775] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592565.960912] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.967188] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:21 2020][8592565.973980] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:21 2020][8592565.980200] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.986909] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:21 2020][8592565.992866] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592565.996201] LNet: Service thread pid 68363 was inactive for 200.36s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:21 2020][8592565.996203] LNet: Skipped 5 previous similar messages [Tue Mar 17 18:01:21 2020][8592566.017781] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.024851] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:21 2020][8592566.032157] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:21 2020][8592566.037905] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:21 2020][8592566.043431] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.050509] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.058257] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:21 2020][8592566.065511] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.073372] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.080350] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:21 2020][8592566.085783] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.092256] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.099827] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:21 2020][8592566.104879] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592566.111145] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:21 2020][8592566.117759] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592566.124033] ll_ost_io02_090 D ffff9c4cb144b0c0 0 90877 2 0x00000080 [Tue Mar 17 18:01:21 2020][8592566.131301] Call Trace: [Tue Mar 17 18:01:21 2020][8592566.133934] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:21 2020][8592566.139251] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:21 2020][8592566.144578] [] schedule+0x29/0x70 [Tue Mar 17 18:01:21 2020][8592566.149721] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:21 2020][8592566.155729] [] ? __find_get_block+0xbc/0x120 [Tue Mar 17 18:01:21 2020][8592566.161830] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:21 2020][8592566.167241] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:21 2020][8592566.173079] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:21 2020][8592566.178400] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:21 2020][8592566.184588] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:21 2020][8592566.189997] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:21 2020][8592566.195404] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:21 2020][8592566.200985] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:21 2020][8592566.206300] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:21 2020][8592566.211618] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:21 2020][8592566.218058] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592566.224157] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:21 2020][8592566.231036] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:21 2020][8592566.238615] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:21 2020][8592566.246875] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:21 2020][8592566.252367] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:21 2020][8592566.259354] [] osd_punch+0x28c/0x6b0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592566.265972] [] ofd_object_punch+0x7df/0xd30 [ofd] [Tue Mar 17 18:01:21 2020][8592566.272507] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:21 2020][8592566.278811] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.285890] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.293636] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:21 2020][8592566.300890] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.308750] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.315717] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:21 2020][8592566.321154] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.327642] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:21 2020][8592566.335216] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:21 2020][8592566.340279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592566.346551] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:21 2020][8592566.353163] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:21 2020][8592566.359431] ll_ost_io00_071 D ffff9c4cb144c100 0 90878 2 0x00000080 [Tue Mar 17 18:01:21 2020][8592566.366708] Call Trace: [Tue Mar 17 18:01:21 2020][8592566.369336] [] schedule+0x29/0x70 [Tue Mar 17 18:01:21 2020][8592566.374479] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:21 2020][8592566.381531] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:21 2020][8592566.387540] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:21 2020][8592566.394781] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:21 2020][8592566.400618] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:21 2020][8592566.407345] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:21 2020][8592566.414655] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:22 2020][8592566.420852] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.427645] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.434960] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.442532] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.449689] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:22 2020][8592566.456499] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:22 2020][8592566.462718] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.469422] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:22 2020][8592566.475377] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.482026] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.489077] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:22 2020][8592566.494832] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:22 2020][8592566.500359] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.507436] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.508207] LustreError: dumping log to /tmp/lustre-log.1584493282.66967 [Tue Mar 17 18:01:22 2020][8592566.522057] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:22 2020][8592566.529311] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.537171] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.544135] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:22 2020][8592566.549567] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.556037] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.563610] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:22 2020][8592566.568663] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592566.574936] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:22 2020][8592566.581549] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592566.587833] ll_ost_io00_072 D ffff9c824ac05140 0 10667 2 0x00000080 [Tue Mar 17 18:01:22 2020][8592566.595102] Call Trace: [Tue Mar 17 18:01:22 2020][8592566.597733] [] schedule+0x29/0x70 [Tue Mar 17 18:01:22 2020][8592566.602881] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.609935] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:22 2020][8592566.615951] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.623178] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:22 2020][8592566.629021] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.635734] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.643049] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:22 2020][8592566.649246] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.656046] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.663365] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.670943] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.678083] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:22 2020][8592566.684878] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:22 2020][8592566.691096] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.697825] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.704474] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.711545] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:22 2020][8592566.718874] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:22 2020][8592566.724621] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:22 2020][8592566.730147] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.737230] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.744976] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:22 2020][8592566.752229] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.760089] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.767053] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:22 2020][8592566.772480] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.778956] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.786528] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:22 2020][8592566.791603] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592566.797872] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:22 2020][8592566.804492] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592566.810758] ll_ost_io00_073 D ffff9c824ac06180 0 10668 2 0x00000080 [Tue Mar 17 18:01:22 2020][8592566.818025] Call Trace: [Tue Mar 17 18:01:22 2020][8592566.820657] [] schedule+0x29/0x70 [Tue Mar 17 18:01:22 2020][8592566.825808] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.832860] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:22 2020][8592566.838876] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.846118] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:22 2020][8592566.851954] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.858670] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.865984] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:22 2020][8592566.872174] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592566.878972] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.886288] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.893861] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592566.901000] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:22 2020][8592566.907795] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:22 2020][8592566.914028] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.920761] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.927410] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.934461] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:22 2020][8592566.939987] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.947066] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.954814] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:22 2020][8592566.962065] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.969926] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.976891] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:22 2020][8592566.982319] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.988811] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592566.996384] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:22 2020][8592567.001446] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592567.007720] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:22 2020][8592567.014340] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592567.020208] LNet: Service thread pid 68490 was inactive for 200.30s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:22 2020][8592567.020210] LNet: Skipped 11 previous similar messages [Tue Mar 17 18:01:22 2020][8592567.038952] ll_ost_io00_074 D ffff9c836cd04100 0 40229 2 0x00000080 [Tue Mar 17 18:01:22 2020][8592567.046222] Call Trace: [Tue Mar 17 18:01:22 2020][8592567.048850] [] schedule+0x29/0x70 [Tue Mar 17 18:01:22 2020][8592567.053995] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.061060] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:22 2020][8592567.067072] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.074296] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:22 2020][8592567.080132] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.086846] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.094162] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:22 2020][8592567.100353] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.107148] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.114467] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.122051] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.129191] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:22 2020][8592567.135988] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:22 2020][8592567.142210] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.148918] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:22 2020][8592567.154586] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:22 2020][8592567.160341] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:22 2020][8592567.165867] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.172947] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.180707] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:22 2020][8592567.187964] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.195826] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.202789] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:22 2020][8592567.208223] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.214691] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.222265] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:22 2020][8592567.227324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592567.233592] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:22 2020][8592567.240213] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:22 2020][8592567.246492] ll_ost_io00_075 D ffff9c836cd030c0 0 40230 2 0x00000080 [Tue Mar 17 18:01:22 2020][8592567.253765] Call Trace: [Tue Mar 17 18:01:22 2020][8592567.256392] [] schedule+0x29/0x70 [Tue Mar 17 18:01:22 2020][8592567.261536] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.268587] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:22 2020][8592567.274595] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.281821] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:22 2020][8592567.287658] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.294371] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.301678] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:22 2020][8592567.307884] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:22 2020][8592567.314684] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.322001] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.329582] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:22 2020][8592567.336722] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:22 2020][8592567.343517] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:22 2020][8592567.349733] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.356463] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.363110] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.370201] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:22 2020][8592567.377535] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:22 2020][8592567.383283] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:22 2020][8592567.388809] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.395890] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:22 2020][8592567.403639] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:22 2020][8592567.410907] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.418770] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.425749] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:23 2020][8592567.431179] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.437654] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.445239] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:23 2020][8592567.450294] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.456560] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:23 2020][8592567.463182] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.469456] ll_ost_io00_076 D ffff9c836cd02080 0 40231 2 0x00000080 [Tue Mar 17 18:01:23 2020][8592567.476725] Call Trace: [Tue Mar 17 18:01:23 2020][8592567.479353] [] schedule+0x29/0x70 [Tue Mar 17 18:01:23 2020][8592567.484498] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.491547] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:23 2020][8592567.497556] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.504798] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:23 2020][8592567.510636] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.517350] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.524664] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:23 2020][8592567.530854] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.532223] LustreError: dumping log to /tmp/lustre-log.1584493283.68808 [Tue Mar 17 18:01:23 2020][8592567.544547] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.551866] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.559440] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.566581] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:23 2020][8592567.573390] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:23 2020][8592567.579610] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.586349] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.592997] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.600047] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:23 2020][8592567.606086] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.613512] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.620592] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.628341] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:23 2020][8592567.635591] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.643454] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.650435] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:23 2020][8592567.655863] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.662339] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.669908] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:23 2020][8592567.674961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.681227] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:23 2020][8592567.687841] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.694114] ll_ost_io00_077 D ffff9c836cd05140 0 40232 2 0x00000080 [Tue Mar 17 18:01:23 2020][8592567.701382] Call Trace: [Tue Mar 17 18:01:23 2020][8592567.704013] [] schedule+0x29/0x70 [Tue Mar 17 18:01:23 2020][8592567.709156] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.716222] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:23 2020][8592567.722233] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.729461] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.736167] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:23 2020][8592567.742357] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.749154] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.756472] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.764045] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.771185] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:23 2020][8592567.777476] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:23 2020][8592567.784272] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:23 2020][8592567.790488] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.797220] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.803865] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.810919] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:23 2020][8592567.816958] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.824391] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.831471] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.839233] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:23 2020][8592567.846491] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.854348] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.861313] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:23 2020][8592567.866742] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.873219] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592567.880785] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:23 2020][8592567.885840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.892106] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:23 2020][8592567.898720] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592567.905009] ll_ost_io00_078 D ffff9c6a1db8e180 0 40233 2 0x00000080 [Tue Mar 17 18:01:23 2020][8592567.912279] Call Trace: [Tue Mar 17 18:01:23 2020][8592567.914907] [] schedule+0x29/0x70 [Tue Mar 17 18:01:23 2020][8592567.920052] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.927105] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:23 2020][8592567.933121] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.940345] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:23 2020][8592567.946182] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.952894] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.960202] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:23 2020][8592567.966407] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592567.973208] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.980523] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.988097] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592567.995237] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:23 2020][8592568.002031] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:23 2020][8592568.008247] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.014978] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.021627] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.028677] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:23 2020][8592568.034221] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.041310] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.049058] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:23 2020][8592568.056309] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.064170] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.071135] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:23 2020][8592568.076564] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.083037] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.090612] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:23 2020][8592568.095671] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592568.101960] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:23 2020][8592568.108576] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592568.114853] ll_ost_io00_079 D ffff9c6a1db8c100 0 40234 2 0x00000080 [Tue Mar 17 18:01:23 2020][8592568.122128] Call Trace: [Tue Mar 17 18:01:23 2020][8592568.124765] [] ? __es_insert_extent+0x119/0x260 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.131994] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:23 2020][8592568.137314] [] schedule+0x29/0x70 [Tue Mar 17 18:01:23 2020][8592568.142454] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:23 2020][8592568.148477] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.156140] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:23 2020][8592568.161996] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:23 2020][8592568.167309] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:23 2020][8592568.173490] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:23 2020][8592568.178898] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:23 2020][8592568.184306] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:23 2020][8592568.189887] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:23 2020][8592568.195208] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:23 2020][8592568.201647] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592568.207743] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:23 2020][8592568.214621] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:23 2020][8592568.220801] [] ? alloc_buffer_head+0x21/0x60 [Tue Mar 17 18:01:23 2020][8592568.226921] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:23 2020][8592568.234501] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.242766] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.249737] [] ? ldiskfs_block_zero_page_range+0x106/0x3e0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.257917] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.264801] [] osd_execute_truncate+0xe0/0x170 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.272287] [] osd_punch+0x188/0x6b0 [osd_ldiskfs] [Tue Mar 17 18:01:23 2020][8592568.278907] [] ofd_object_punch+0x7df/0xd30 [ofd] [Tue Mar 17 18:01:23 2020][8592568.285441] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:23 2020][8592568.291758] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.298843] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.306591] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:23 2020][8592568.313842] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.321703] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.328667] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:23 2020][8592568.334097] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.340570] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:23 2020][8592568.348142] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:23 2020][8592568.353204] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592568.359492] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:23 2020][8592568.366108] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:23 2020][8592568.372403] ll_ost_io00_080 D ffff9c6a1db89040 0 40235 2 0x00000080 [Tue Mar 17 18:01:23 2020][8592568.379676] Call Trace: [Tue Mar 17 18:01:23 2020][8592568.382307] [] schedule+0x29/0x70 [Tue Mar 17 18:01:23 2020][8592568.387458] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:23 2020][8592568.394508] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:23 2020][8592568.400518] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:23 2020][8592568.407758] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:23 2020][8592568.413597] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.420324] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.427633] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:24 2020][8592568.433838] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.440631] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.447946] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.455521] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.462660] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:24 2020][8592568.469452] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:24 2020][8592568.475671] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.482372] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:24 2020][8592568.487898] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.494993] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.502742] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:24 2020][8592568.509994] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.517856] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.524822] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:24 2020][8592568.530259] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.536732] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.544304] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:24 2020][8592568.549358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592568.555646] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:24 2020][8592568.556223] LustreError: dumping log to /tmp/lustre-log.1584493284.68900 [Tue Mar 17 18:01:24 2020][8592568.569134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592568.575400] ll_ost_io00_081 D ffff9c5e79894100 0 40236 2 0x00000080 [Tue Mar 17 18:01:24 2020][8592568.582677] Call Trace: [Tue Mar 17 18:01:24 2020][8592568.585305] [] schedule+0x29/0x70 [Tue Mar 17 18:01:24 2020][8592568.590447] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.597500] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:24 2020][8592568.603508] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.610735] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:24 2020][8592568.616580] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.623292] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.630599] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:24 2020][8592568.636789] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.643588] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.650905] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.658484] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.665627] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:24 2020][8592568.672419] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:24 2020][8592568.678637] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.685355] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:24 2020][8592568.691024] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:24 2020][8592568.696783] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:01:24 2020][8592568.702446] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:24 2020][8592568.707972] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.715053] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.722799] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:24 2020][8592568.730051] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.737913] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.744880] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:24 2020][8592568.750678] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.757153] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.764726] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:24 2020][8592568.769787] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592568.776062] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:24 2020][8592568.782681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592568.788949] ll_ost_io00_082 D ffff9c6a336e2080 0 40237 2 0x00000080 [Tue Mar 17 18:01:24 2020][8592568.796226] Call Trace: [Tue Mar 17 18:01:24 2020][8592568.798856] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:24 2020][8592568.805476] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:24 2020][8592568.811852] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:24 2020][8592568.817169] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:24 2020][8592568.822670] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.829638] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.837034] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:24 2020][8592568.843054] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:24 2020][8592568.849411] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:24 2020][8592568.856029] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:24 2020][8592568.861524] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:24 2020][8592568.866759] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:24 2020][8592568.873915] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:24 2020][8592568.879142] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:24 2020][8592568.884808] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.892210] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:24 2020][8592568.899179] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.906506] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.913390] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592568.920705] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:24 2020][8592568.927587] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:24 2020][8592568.933802] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.940546] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.947190] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.954242] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:24 2020][8592568.959768] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.966847] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.974594] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:24 2020][8592568.981848] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.989708] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592568.996673] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:24 2020][8592569.002111] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.008599] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.016172] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:24 2020][8592569.021227] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592569.027499] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:24 2020][8592569.034113] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592569.040387] ll_ost_io00_083 D ffff9c690a5c8000 0 40238 2 0x00000080 [Tue Mar 17 18:01:24 2020][8592569.047655] Call Trace: [Tue Mar 17 18:01:24 2020][8592569.050286] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:24 2020][8592569.056300] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:24 2020][8592569.062921] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:24 2020][8592569.068219] LNet: Service thread pid 40245 was inactive for 200.36s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:24 2020][8592569.068221] LNet: Skipped 50 previous similar messages [Tue Mar 17 18:01:24 2020][8592569.087628] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:24 2020][8592569.092941] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:24 2020][8592569.098436] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.105404] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.112798] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:24 2020][8592569.119070] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.127423] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:24 2020][8592569.133798] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.141048] [] ? ldiskfs_es_insert_extent+0xdb/0x1c0 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.148704] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:24 2020][8592569.154293] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.161954] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.169614] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.177535] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.184938] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.191995] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.200185] [] ? osd_trunc_lock+0x86/0x210 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.207333] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.214559] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.221890] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:24 2020][8592569.228685] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:24 2020][8592569.234902] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.241633] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.248281] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.255353] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:24 2020][8592569.262664] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:24 2020][8592569.268418] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:24 2020][8592569.274978] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.282418] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.289501] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.297249] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:24 2020][8592569.304500] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.312361] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.319326] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:24 2020][8592569.324764] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.331237] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:24 2020][8592569.338807] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:24 2020][8592569.343862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592569.350153] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:24 2020][8592569.356768] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:24 2020][8592569.363042] ll_ost_io00_084 D ffff9c690a5cd140 0 40239 2 0x00000080 [Tue Mar 17 18:01:24 2020][8592569.370342] Call Trace: [Tue Mar 17 18:01:24 2020][8592569.372974] [] schedule+0x29/0x70 [Tue Mar 17 18:01:24 2020][8592569.378124] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:24 2020][8592569.385176] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:24 2020][8592569.391185] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:24 2020][8592569.398413] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:24 2020][8592569.405125] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:24 2020][8592569.412896] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:24 2020][8592569.419083] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:25 2020][8592569.425880] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.433197] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.440778] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.447920] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:25 2020][8592569.454193] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:25 2020][8592569.460988] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:25 2020][8592569.467205] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.473943] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.480607] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.487679] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:25 2020][8592569.494993] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:25 2020][8592569.500747] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:25 2020][8592569.507299] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.514724] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.521804] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.529550] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:25 2020][8592569.536804] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.544666] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.551645] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:25 2020][8592569.557075] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.563547] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.571121] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:25 2020][8592569.576183] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592569.580224] LustreError: dumping log to /tmp/lustre-log.1584493285.66968 [Tue Mar 17 18:01:25 2020][8592569.589336] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:25 2020][8592569.595968] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592569.602233] ll_ost_io00_085 D ffff9c690a5cc100 0 40240 2 0x00000080 [Tue Mar 17 18:01:25 2020][8592569.609514] Call Trace: [Tue Mar 17 18:01:25 2020][8592569.612150] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:25 2020][8592569.618771] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:25 2020][8592569.625132] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:25 2020][8592569.630444] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:25 2020][8592569.635948] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.642913] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.650309] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:25 2020][8592569.656327] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:25 2020][8592569.662689] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:25 2020][8592569.669311] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.676030] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:25 2020][8592569.683165] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:25 2020][8592569.688398] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:25 2020][8592569.694062] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.701463] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:25 2020][8592569.708434] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.715750] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.722629] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.729942] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:25 2020][8592569.736825] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:25 2020][8592569.743055] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.749788] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.756436] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.763505] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:25 2020][8592569.770819] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:25 2020][8592569.776565] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:25 2020][8592569.782086] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.789165] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.796910] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:25 2020][8592569.804165] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.812038] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.818999] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:25 2020][8592569.824435] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.830909] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592569.838481] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:25 2020][8592569.843542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592569.849817] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:25 2020][8592569.856439] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592569.862714] ll_ost_io00_086 D ffff9c690a5ce180 0 40241 2 0x00000080 [Tue Mar 17 18:01:25 2020][8592569.869989] Call Trace: [Tue Mar 17 18:01:25 2020][8592569.872629] [] ? __begin_wqe+0x4b/0x140 [mlx5_ib] [Tue Mar 17 18:01:25 2020][8592569.879162] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:25 2020][8592569.884498] [] schedule+0x29/0x70 [Tue Mar 17 18:01:25 2020][8592569.889640] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:25 2020][8592569.895655] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:25 2020][8592569.900976] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:25 2020][8592569.907155] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:25 2020][8592569.912555] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:25 2020][8592569.917953] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:25 2020][8592569.923961] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:25 2020][8592569.929297] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:25 2020][8592569.936172] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592569.942275] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:25 2020][8592569.947857] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:25 2020][8592569.954743] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.961532] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:25 2020][8592569.969113] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.977381] [] ldiskfs_quota_write+0xc6/0x1e0 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592569.984431] [] qtree_write_dquot+0xd5/0x180 [Tue Mar 17 18:01:25 2020][8592569.990443] [] v2_write_dquot+0x2b/0x30 [Tue Mar 17 18:01:25 2020][8592569.996116] [] dquot_commit+0xbd/0xd0 [Tue Mar 17 18:01:25 2020][8592570.001613] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.008581] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.015975] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:25 2020][8592570.021994] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:25 2020][8592570.028355] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:25 2020][8592570.034979] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.041681] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:25 2020][8592570.048823] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:25 2020][8592570.054055] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:25 2020][8592570.059735] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.067138] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:25 2020][8592570.074109] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.081423] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.088304] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.095618] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:25 2020][8592570.102500] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:25 2020][8592570.108716] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.115447] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.122095] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.129145] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:25 2020][8592570.135204] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.142634] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.149718] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.157464] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:25 2020][8592570.164719] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.172578] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.179542] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:25 2020][8592570.184979] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.191452] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.199025] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:25 2020][8592570.204093] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592570.210362] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:25 2020][8592570.216981] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592570.223248] ll_ost_io00_087 D ffff9c690a5c9040 0 40242 2 0x00000080 [Tue Mar 17 18:01:25 2020][8592570.230517] Call Trace: [Tue Mar 17 18:01:25 2020][8592570.233146] [] schedule+0x29/0x70 [Tue Mar 17 18:01:25 2020][8592570.238298] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:25 2020][8592570.245350] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:25 2020][8592570.251364] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:25 2020][8592570.258606] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:25 2020][8592570.264444] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:25 2020][8592570.271158] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.278473] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:25 2020][8592570.284656] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:25 2020][8592570.291454] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.298770] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.306350] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:25 2020][8592570.313490] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:25 2020][8592570.320286] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:25 2020][8592570.326518] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.333222] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:25 2020][8592570.338749] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.345828] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.353573] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:25 2020][8592570.360829] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.368687] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.375654] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:25 2020][8592570.381092] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.387565] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:25 2020][8592570.395138] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:25 2020][8592570.400214] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592570.406483] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:25 2020][8592570.413117] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:25 2020][8592570.419385] ll_ost_io00_088 D ffff9c690a5ca080 0 40243 2 0x00000080 [Tue Mar 17 18:01:26 2020][8592570.426654] Call Trace: [Tue Mar 17 18:01:26 2020][8592570.429284] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:26 2020][8592570.435918] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:26 2020][8592570.442273] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:26 2020][8592570.447587] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:26 2020][8592570.453096] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.460065] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.467475] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:26 2020][8592570.473744] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:26 2020][8592570.480112] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.487338] [] ? osd_read_unlock+0x58/0x90 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.494478] [] ? lquota_disk_read+0xf2/0x390 [lquota] [Tue Mar 17 18:01:26 2020][8592570.501355] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:26 2020][8592570.506935] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.514596] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.522272] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:01:26 2020][8592570.528809] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.536211] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.543267] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.551444] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:26 2020][8592570.558415] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.565642] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.572956] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:26 2020][8592570.579749] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:26 2020][8592570.585968] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.592715] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.599363] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.604225] LustreError: dumping log to /tmp/lustre-log.1584493286.68770 [Tue Mar 17 18:01:26 2020][8592570.613285] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:26 2020][8592570.619332] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.626759] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.633842] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.641589] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:26 2020][8592570.648840] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.656702] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.663681] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:26 2020][8592570.669110] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.675587] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.683157] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:26 2020][8592570.688218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592570.694487] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:26 2020][8592570.701105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592570.707372] ll_ost_io00_089 D ffff9c5ae95a30c0 0 40244 2 0x00000080 [Tue Mar 17 18:01:26 2020][8592570.714640] Call Trace: [Tue Mar 17 18:01:26 2020][8592570.717270] [] schedule+0x29/0x70 [Tue Mar 17 18:01:26 2020][8592570.722421] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:26 2020][8592570.729488] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:26 2020][8592570.735497] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:26 2020][8592570.742723] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:26 2020][8592570.748557] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:26 2020][8592570.755274] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.762589] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:26 2020][8592570.768780] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:26 2020][8592570.775577] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.782894] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.790490] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592570.797632] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:26 2020][8592570.804426] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:26 2020][8592570.810645] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.817382] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.824032] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.831099] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:26 2020][8592570.838414] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:26 2020][8592570.844169] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:26 2020][8592570.850727] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.858170] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.865250] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.872997] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:26 2020][8592570.880251] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.888110] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.895077] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:26 2020][8592570.900513] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.906987] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592570.914559] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:26 2020][8592570.919622] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592570.925911] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:26 2020][8592570.932524] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592570.938792] ll_ost_io01_093 D ffff9c6a39cdc100 0 40245 2 0x00000080 [Tue Mar 17 18:01:26 2020][8592570.946066] Call Trace: [Tue Mar 17 18:01:26 2020][8592570.948698] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:26 2020][8592570.954019] [] schedule+0x29/0x70 [Tue Mar 17 18:01:26 2020][8592570.959168] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:26 2020][8592570.965184] [] ? __radix_tree_lookup+0x84/0xf0 [Tue Mar 17 18:01:26 2020][8592570.971456] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:26 2020][8592570.976776] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:26 2020][8592570.982970] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:26 2020][8592570.988372] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:26 2020][8592570.993773] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:26 2020][8592570.999788] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:26 2020][8592571.005109] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:26 2020][8592571.011983] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592571.018085] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:26 2020][8592571.023667] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.030552] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.037345] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.044939] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.053215] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.061049] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.067685] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:01:26 2020][8592571.074509] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.082457] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.089623] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.096344] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:01:26 2020][8592571.103146] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.110214] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.117384] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:26 2020][8592571.124697] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:26 2020][8592571.130882] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:26 2020][8592571.137679] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:26 2020][8592571.143896] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.150626] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.157276] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.164325] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:26 2020][8592571.169844] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.176923] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.184673] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:26 2020][8592571.191938] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.199802] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.206766] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:26 2020][8592571.212208] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.218683] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.226259] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:26 2020][8592571.231319] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592571.237586] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:26 2020][8592571.244211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:26 2020][8592571.250495] ll_ost_io00_090 D ffff9c6a3622d140 0 40246 2 0x00000080 [Tue Mar 17 18:01:26 2020][8592571.257768] Call Trace: [Tue Mar 17 18:01:26 2020][8592571.260395] [] schedule+0x29/0x70 [Tue Mar 17 18:01:26 2020][8592571.265540] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.272589] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:26 2020][8592571.278599] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.285824] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:26 2020][8592571.291659] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.298373] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.305680] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:26 2020][8592571.311869] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:26 2020][8592571.318669] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.325985] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.333558] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:26 2020][8592571.340698] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:26 2020][8592571.347493] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:26 2020][8592571.353709] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.360445] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.367114] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.374184] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:26 2020][8592571.381495] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:26 2020][8592571.387256] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:26 2020][8592571.392778] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.399859] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:26 2020][8592571.407606] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:26 2020][8592571.414873] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.422736] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.429701] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:27 2020][8592571.435109] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:27 2020][8592571.441729] [] ? _raw_spin_lock+0x20/0x30 [Tue Mar 17 18:01:27 2020][8592571.447609] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.454084] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.461669] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592571.466726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.473003] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592571.479621] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.485910] ll_ost_io00_091 D ffff9c6a36228000 0 40247 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592571.493181] Call Trace: [Tue Mar 17 18:01:27 2020][8592571.495811] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592571.500973] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.508021] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:27 2020][8592571.514031] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.521259] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:27 2020][8592571.527187] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.533918] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.541234] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:27 2020][8592571.547426] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.554221] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.561542] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.569120] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.576259] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:27 2020][8592571.583054] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:27 2020][8592571.589281] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.596018] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.602665] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.609758] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:27 2020][8592571.617068] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:27 2020][8592571.622819] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:27 2020][8592571.628234] LustreError: dumping log to /tmp/lustre-log.1584493287.68759 [Tue Mar 17 18:01:27 2020][8592571.635223] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.642309] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.650058] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:27 2020][8592571.657309] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.665170] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.672137] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:27 2020][8592571.677572] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.684044] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.691617] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592571.696670] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.702936] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592571.709549] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.715816] ll_ost_io01_094 S ffff9c5a2862c100 0 40248 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592571.723093] Call Trace: [Tue Mar 17 18:01:27 2020][8592571.725726] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:27 2020][8592571.731821] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592571.737007] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.743889] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:27 2020][8592571.749670] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.756143] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.763716] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592571.768777] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.775043] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592571.781656] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.787932] ll_ost_io03_075 D ffff9c79ca36e180 0 40250 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592571.795226] Call Trace: [Tue Mar 17 18:01:27 2020][8592571.797853] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592571.803000] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.810058] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:27 2020][8592571.817197] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:27 2020][8592571.823210] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:27 2020][8592571.829397] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:27 2020][8592571.836191] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:27 2020][8592571.842409] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.849138] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.855786] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.862871] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:27 2020][8592571.870178] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:27 2020][8592571.875923] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:27 2020][8592571.881441] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.888523] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.896271] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:27 2020][8592571.903522] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.911383] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.918348] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:27 2020][8592571.923786] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.930272] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592571.937840] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592571.942901] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.949177] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592571.955798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592571.962071] ll_ost_io03_076 D ffff9c6a3622c100 0 40251 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592571.969340] Call Trace: [Tue Mar 17 18:01:27 2020][8592571.971971] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592571.977121] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.984185] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:27 2020][8592571.990196] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592571.997425] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:27 2020][8592572.003268] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.009982] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.017296] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:27 2020][8592572.023486] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.030284] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.037602] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.045181] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.052338] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:27 2020][8592572.059134] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:27 2020][8592572.065351] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.072084] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.078730] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.085781] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:27 2020][8592572.091306] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.098387] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.106136] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:27 2020][8592572.113387] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.121249] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.128230] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:27 2020][8592572.133659] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.140131] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.147703] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592572.152755] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.159022] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592572.165634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.171902] ldlm_cb02_006 S ffff9c5a2862d140 0 40400 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592572.179180] Call Trace: [Tue Mar 17 18:01:27 2020][8592572.181807] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:27 2020][8592572.187901] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592572.193094] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.199973] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:27 2020][8592572.205747] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.212194] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:01:27 2020][8592572.217891] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.225461] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592572.230514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.236783] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592572.243416] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.249685] ll_ost_io03_077 S ffff9c84b1d14100 0 40772 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592572.256956] Call Trace: [Tue Mar 17 18:01:27 2020][8592572.259582] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:27 2020][8592572.265674] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592572.270845] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.277723] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:27 2020][8592572.283504] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.289980] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:27 2020][8592572.297551] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:27 2020][8592572.302606] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.308895] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:27 2020][8592572.315508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:27 2020][8592572.321775] ll_ost_io03_078 D ffff9c84b1d11040 0 40773 2 0x00000080 [Tue Mar 17 18:01:27 2020][8592572.329044] Call Trace: [Tue Mar 17 18:01:27 2020][8592572.331672] [] schedule+0x29/0x70 [Tue Mar 17 18:01:27 2020][8592572.336816] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.343869] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:27 2020][8592572.349885] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.357144] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:27 2020][8592572.362980] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.369710] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.377016] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:27 2020][8592572.383198] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:27 2020][8592572.389997] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.397312] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.404887] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:27 2020][8592572.412027] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:27 2020][8592572.418836] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:28 2020][8592572.425055] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.431802] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.438452] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.445519] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:28 2020][8592572.452835] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:28 2020][8592572.458589] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:28 2020][8592572.464116] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.471196] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.478943] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:28 2020][8592572.486195] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.494056] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.501038] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:28 2020][8592572.506466] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.512940] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.520512] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:28 2020][8592572.525563] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.531832] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:28 2020][8592572.538453] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.544728] ll_ost_io03_079 D ffff9c84b1d12080 0 40774 2 0x00000080 [Tue Mar 17 18:01:28 2020][8592572.552003] Call Trace: [Tue Mar 17 18:01:28 2020][8592572.554634] [] schedule+0x29/0x70 [Tue Mar 17 18:01:28 2020][8592572.559786] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.566851] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:28 2020][8592572.572863] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.580088] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:28 2020][8592572.585933] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.592646] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.599961] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:28 2020][8592572.606151] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.612949] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.620266] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.627851] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.634995] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:28 2020][8592572.641788] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:28 2020][8592572.648009] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.652234] LustreError: dumping log to /tmp/lustre-log.1584493288.68840 [Tue Mar 17 18:01:28 2020][8592572.661589] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:28 2020][8592572.667130] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.674213] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.681959] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:28 2020][8592572.689216] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.697071] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.704039] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:28 2020][8592572.709476] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.715950] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.723523] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:28 2020][8592572.728585] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.734858] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:28 2020][8592572.741478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.747744] ll_ost_io03_080 D ffff9c5c85210000 0 40775 2 0x00000080 [Tue Mar 17 18:01:28 2020][8592572.755014] Call Trace: [Tue Mar 17 18:01:28 2020][8592572.757642] [] schedule+0x29/0x70 [Tue Mar 17 18:01:28 2020][8592572.762787] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.769852] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:28 2020][8592572.775862] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.783091] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.789803] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.797545] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:28 2020][8592572.803735] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.810532] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.817848] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.825435] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592572.832580] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:28 2020][8592572.838853] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:28 2020][8592572.845649] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:28 2020][8592572.851867] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.858567] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:28 2020][8592572.864524] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.871172] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.878222] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:28 2020][8592572.884255] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.891703] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.898785] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.906533] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:28 2020][8592572.913786] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.921645] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.928611] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:28 2020][8592572.934048] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.940522] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592572.948093] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:28 2020][8592572.953148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.959434] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:28 2020][8592572.966052] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592572.972325] ll_ost_io03_081 D ffff9c5ae95db0c0 0 40776 2 0x00000080 [Tue Mar 17 18:01:28 2020][8592572.979595] Call Trace: [Tue Mar 17 18:01:28 2020][8592572.982222] [] schedule+0x29/0x70 [Tue Mar 17 18:01:28 2020][8592572.987366] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:28 2020][8592572.994418] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:28 2020][8592573.000434] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.007661] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:28 2020][8592573.013591] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.020321] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.027628] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:28 2020][8592573.033810] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.040609] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.047926] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.055496] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.062636] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:28 2020][8592573.069433] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:28 2020][8592573.075649] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.082379] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.089029] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.096094] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:28 2020][8592573.102126] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.109551] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.116632] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.124380] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:28 2020][8592573.131632] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.139491] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.146458] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:28 2020][8592573.151895] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.158384] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.164235] LNet: Service thread pid 68386 was inactive for 200.46s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:28 2020][8592573.164237] LNet: Skipped 72 previous similar messages [Tue Mar 17 18:01:28 2020][8592573.184305] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:28 2020][8592573.189358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592573.195626] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:28 2020][8592573.202246] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592573.208521] ll_ost_io02_091 D ffff9c5ae94da080 0 40777 2 0x00000080 [Tue Mar 17 18:01:28 2020][8592573.215796] Call Trace: [Tue Mar 17 18:01:28 2020][8592573.218428] [] schedule+0x29/0x70 [Tue Mar 17 18:01:28 2020][8592573.223580] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.230650] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.237786] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:28 2020][8592573.243798] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:28 2020][8592573.249985] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:28 2020][8592573.256781] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:28 2020][8592573.262996] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.269730] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.276376] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.283428] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:28 2020][8592573.288954] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.296049] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.303798] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:28 2020][8592573.311051] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.318912] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.325875] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:28 2020][8592573.331305] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.337779] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:28 2020][8592573.345350] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:28 2020][8592573.350404] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592573.356711] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:28 2020][8592573.363324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:28 2020][8592573.369592] ll_ost_io02_092 D ffff9c6a05388000 0 40778 2 0x00000080 [Tue Mar 17 18:01:28 2020][8592573.376870] Call Trace: [Tue Mar 17 18:01:28 2020][8592573.379505] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:28 2020][8592573.387162] [] schedule+0x29/0x70 [Tue Mar 17 18:01:28 2020][8592573.392312] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.399361] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:28 2020][8592573.405371] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.412599] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:28 2020][8592573.419325] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.427087] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:29 2020][8592573.433292] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:29 2020][8592573.440092] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.447409] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.454981] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.462137] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:29 2020][8592573.468415] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:29 2020][8592573.475208] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:29 2020][8592573.481426] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.488125] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:29 2020][8592573.493660] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.500743] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.508488] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:29 2020][8592573.515742] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.523603] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.530566] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:29 2020][8592573.535996] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.542470] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.550040] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:29 2020][8592573.555109] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592573.561377] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:29 2020][8592573.567992] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592573.574264] ll_ost_io02_093 D ffff9c5ae9672080 0 40779 2 0x00000080 [Tue Mar 17 18:01:29 2020][8592573.581534] Call Trace: [Tue Mar 17 18:01:29 2020][8592573.584162] [] schedule+0x29/0x70 [Tue Mar 17 18:01:29 2020][8592573.589304] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:29 2020][8592573.595750] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592573.601851] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:29 2020][8592573.607604] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:29 2020][8592573.613113] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.619563] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.626015] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.633420] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.640478] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.647443] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.654583] [] ? iova_rcache_get+0xba/0x140 [Tue Mar 17 18:01:29 2020][8592573.660614] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:29 2020][8592573.667932] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.675001] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:29 2020][8592573.676241] LustreError: dumping log to /tmp/lustre-log.1584493289.68721 [Tue Mar 17 18:01:29 2020][8592573.689394] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:29 2020][8592573.696454] [] ? kiblnd_check_sends_locked+0x180/0x470 [ko2iblnd] [Tue Mar 17 18:01:29 2020][8592573.704373] [] ? kiblnd_queue_tx+0x36/0x50 [ko2iblnd] [Tue Mar 17 18:01:29 2020][8592573.711255] [] ? kiblnd_launch_tx+0xd2/0xc20 [ko2iblnd] [Tue Mar 17 18:01:29 2020][8592573.718324] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:29 2020][8592573.724950] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:29 2020][8592573.731339] [] ? null_alloc_rs+0x186/0x340 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.738043] [] ofd_preprw_write.isra.31+0x85/0xea0 [ofd] [Tue Mar 17 18:01:29 2020][8592573.745214] [] ? lustre_msg_add_version+0x27/0xa0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.752549] [] ofd_preprw+0x422/0x11b0 [ofd] [Tue Mar 17 18:01:29 2020][8592573.758680] [] tgt_brw_write+0xc7c/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.765327] [] ? lustre_msg_buf_v2+0x1e0/0x1e0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.772406] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.779053] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.786106] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:29 2020][8592573.792121] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:29 2020][8592573.797650] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.804728] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.812473] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:29 2020][8592573.819744] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.827603] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.834570] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:29 2020][8592573.839997] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.846474] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592573.854044] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:29 2020][8592573.859097] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592573.865370] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:29 2020][8592573.871984] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592573.878249] ll_ost_io02_094 D ffff9c6c34002080 0 40780 2 0x00000080 [Tue Mar 17 18:01:29 2020][8592573.885536] Call Trace: [Tue Mar 17 18:01:29 2020][8592573.888168] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:29 2020][8592573.893486] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:29 2020][8592573.900097] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:29 2020][8592573.906452] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:29 2020][8592573.911768] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:29 2020][8592573.917267] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.924236] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.931628] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:29 2020][8592573.937904] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.944805] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.953165] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.960393] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.968047] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:29 2020][8592573.973634] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.981295] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.989219] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592573.996626] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.003684] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.011870] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:29 2020][8592574.018854] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.026085] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.033397] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:29 2020][8592574.040193] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:29 2020][8592574.046410] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.053139] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.059788] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.066858] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:29 2020][8592574.074171] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:29 2020][8592574.079925] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:29 2020][8592574.085459] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.092541] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.100286] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:29 2020][8592574.107540] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.115403] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.122368] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:29 2020][8592574.127803] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.134277] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.141848] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:29 2020][8592574.146901] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592574.153183] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:29 2020][8592574.159797] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592574.166065] ll_ost_io02_095 D ffff9c6a2cedd140 0 40781 2 0x00000080 [Tue Mar 17 18:01:29 2020][8592574.173340] Call Trace: [Tue Mar 17 18:01:29 2020][8592574.175977] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.183631] [] schedule+0x29/0x70 [Tue Mar 17 18:01:29 2020][8592574.188775] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:29 2020][8592574.195828] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:29 2020][8592574.201842] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:29 2020][8592574.209069] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:29 2020][8592574.215783] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.223539] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:29 2020][8592574.229726] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:29 2020][8592574.236521] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.243837] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.251409] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:29 2020][8592574.258552] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:29 2020][8592574.264826] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:29 2020][8592574.271620] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:29 2020][8592574.277837] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.284570] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.291233] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.298300] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:29 2020][8592574.305606] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:29 2020][8592574.311351] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:29 2020][8592574.317905] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.325340] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.332418] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.340165] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:29 2020][8592574.347417] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.355277] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.362276] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:29 2020][8592574.367706] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.374180] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:29 2020][8592574.381754] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:29 2020][8592574.386815] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592574.393088] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:29 2020][8592574.399701] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:29 2020][8592574.405968] ll_ost_io01_095 D ffff9c6c34001040 0 40782 2 0x00000080 [Tue Mar 17 18:01:29 2020][8592574.413245] Call Trace: [Tue Mar 17 18:01:29 2020][8592574.415873] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:29 2020][8592574.422503] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:30 2020][8592574.428871] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:30 2020][8592574.434186] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:30 2020][8592574.439680] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.446647] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.454043] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:30 2020][8592574.460065] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:30 2020][8592574.466425] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:30 2020][8592574.473045] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.479764] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:30 2020][8592574.486896] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:30 2020][8592574.492123] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:30 2020][8592574.497797] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.505196] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.512167] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.519483] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.526364] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.533676] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:30 2020][8592574.540557] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:30 2020][8592574.546788] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.553520] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.560169] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.567241] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:30 2020][8592574.574553] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:30 2020][8592574.580307] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:30 2020][8592574.585827] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.592905] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.600652] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:30 2020][8592574.607905] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.615781] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.622747] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:30 2020][8592574.628177] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.634650] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.642224] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:30 2020][8592574.647285] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592574.653557] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:30 2020][8592574.660171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592574.666444] ll_ost_io02_096 D ffff9c6c34006180 0 40783 2 0x00000080 [Tue Mar 17 18:01:30 2020][8592574.673714] Call Trace: [Tue Mar 17 18:01:30 2020][8592574.676343] [] schedule+0x29/0x70 [Tue Mar 17 18:01:30 2020][8592574.681486] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.688552] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:30 2020][8592574.694564] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.700236] LustreError: dumping log to /tmp/lustre-log.1584493290.68371 [Tue Mar 17 18:01:30 2020][8592574.708664] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.715393] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.722701] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:30 2020][8592574.728885] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.735680] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.742998] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.750571] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.757712] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:30 2020][8592574.764506] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:30 2020][8592574.770721] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.777426] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:30 2020][8592574.783464] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.790898] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.797979] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.805726] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:30 2020][8592574.812977] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.820856] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.827822] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:30 2020][8592574.833257] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.839731] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.847304] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:30 2020][8592574.852364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592574.858631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:30 2020][8592574.865251] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592574.871517] ll_ost_io02_097 D ffff9c6d7768e180 0 40784 2 0x00000080 [Tue Mar 17 18:01:30 2020][8592574.878804] Call Trace: [Tue Mar 17 18:01:30 2020][8592574.881441] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.889104] [] schedule+0x29/0x70 [Tue Mar 17 18:01:30 2020][8592574.894254] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.901306] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:30 2020][8592574.907322] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.914550] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.921263] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.929004] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:30 2020][8592574.935188] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592574.942000] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.949320] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.956899] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592574.964040] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:30 2020][8592574.970313] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:30 2020][8592574.977108] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:30 2020][8592574.983326] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592574.990028] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:30 2020][8592574.995696] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:30 2020][8592575.001450] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:30 2020][8592575.008027] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.015453] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.022533] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.030279] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:30 2020][8592575.037532] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.045396] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.052357] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:30 2020][8592575.057786] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.064258] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.071831] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:30 2020][8592575.076899] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592575.083168] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:30 2020][8592575.089789] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592575.096055] ll_ost_io01_096 D ffff9c6a2ced8000 0 40785 2 0x00000080 [Tue Mar 17 18:01:30 2020][8592575.103325] Call Trace: [Tue Mar 17 18:01:30 2020][8592575.105962] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.113622] [] schedule+0x29/0x70 [Tue Mar 17 18:01:30 2020][8592575.118766] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:30 2020][8592575.125816] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:30 2020][8592575.131826] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592575.139071] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:30 2020][8592575.145783] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.153525] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:30 2020][8592575.159716] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:30 2020][8592575.166514] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.173830] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.181403] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.188543] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:30 2020][8592575.194817] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:30 2020][8592575.201611] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:30 2020][8592575.207846] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.214578] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.221228] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.228301] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:30 2020][8592575.235611] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:30 2020][8592575.241363] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:30 2020][8592575.247922] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.255348] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.262428] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.270174] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:30 2020][8592575.277444] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.285306] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.292271] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:30 2020][8592575.297707] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.304181] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:30 2020][8592575.311752] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:30 2020][8592575.316805] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592575.323073] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:30 2020][8592575.329694] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:30 2020][8592575.335968] ll_ost_io01_097 D ffff9c6a2ceda080 0 40786 2 0x00000080 [Tue Mar 17 18:01:30 2020][8592575.343253] Call Trace: [Tue Mar 17 18:01:30 2020][8592575.345881] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:30 2020][8592575.352497] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:30 2020][8592575.358857] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:30 2020][8592575.364169] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:30 2020][8592575.369661] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.376631] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:30 2020][8592575.384027] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:30 2020][8592575.390047] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:30 2020][8592575.396406] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:30 2020][8592575.403023] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:30 2020][8592575.408532] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:30 2020][8592575.413760] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:30 2020][8592575.420918] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:30 2020][8592575.426145] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:31 2020][8592575.431814] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.439239] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.446208] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.453519] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.460417] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.467747] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:31 2020][8592575.474631] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:31 2020][8592575.480849] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.487577] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.494229] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.501286] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:31 2020][8592575.506803] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.513883] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.521632] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:31 2020][8592575.528884] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.536744] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.543725] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:31 2020][8592575.549154] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.555631] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.563202] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:31 2020][8592575.568261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.574528] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:31 2020][8592575.581148] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.587417] ll_ost_io02_098 D ffff9c4b973a5140 0 40787 2 0x00000080 [Tue Mar 17 18:01:31 2020][8592575.594694] Call Trace: [Tue Mar 17 18:01:31 2020][8592575.597329] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.604984] [] schedule+0x29/0x70 [Tue Mar 17 18:01:31 2020][8592575.610149] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.617205] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:31 2020][8592575.623237] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.630466] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.637176] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.644919] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:31 2020][8592575.651101] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.657899] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.665214] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.672805] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.679945] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:31 2020][8592575.686223] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:31 2020][8592575.693023] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:31 2020][8592575.699242] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.705980] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.712629] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.719679] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:31 2020][8592575.724242] LustreError: dumping log to /tmp/lustre-log.1584493291.68393 [Tue Mar 17 18:01:31 2020][8592575.732085] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.739168] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.746915] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:31 2020][8592575.754167] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.762028] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.768994] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:31 2020][8592575.774428] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.780904] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.788475] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:31 2020][8592575.793537] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.799827] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:31 2020][8592575.806441] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.812707] ll_ost_io02_099 D ffff9c4b973a30c0 0 40788 2 0x00000080 [Tue Mar 17 18:01:31 2020][8592575.819975] Call Trace: [Tue Mar 17 18:01:31 2020][8592575.822605] [] schedule+0x29/0x70 [Tue Mar 17 18:01:31 2020][8592575.827746] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.834799] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:31 2020][8592575.840817] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.848044] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.854754] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.862512] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:31 2020][8592575.868698] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592575.875495] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.882812] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.890384] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592575.897525] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:31 2020][8592575.903798] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:01:31 2020][8592575.910333] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:31 2020][8592575.916637] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.923715] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.931463] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:31 2020][8592575.938729] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.946585] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.953550] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:31 2020][8592575.958986] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.965463] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592575.973033] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:31 2020][8592575.978084] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.984352] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:31 2020][8592575.990972] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592575.997253] ll_ost_io01_098 D ffff9c4b973a0000 0 40789 2 0x00000080 [Tue Mar 17 18:01:31 2020][8592576.004524] Call Trace: [Tue Mar 17 18:01:31 2020][8592576.007154] [] schedule+0x29/0x70 [Tue Mar 17 18:01:31 2020][8592576.012307] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.019357] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:31 2020][8592576.025373] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.032604] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.039313] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.047057] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:31 2020][8592576.053246] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.060058] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.067379] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.074950] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.082090] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:31 2020][8592576.088366] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:31 2020][8592576.095159] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:31 2020][8592576.101377] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.108106] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.114756] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.121824] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:31 2020][8592576.129153] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:31 2020][8592576.134902] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:31 2020][8592576.141464] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.148895] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.155978] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.163723] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:31 2020][8592576.170974] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.178836] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.185801] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:31 2020][8592576.191232] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.197717] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.205285] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:31 2020][8592576.210347] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592576.216620] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:31 2020][8592576.223233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592576.229499] ll_ost_io01_099 D ffff9c4b973a2080 0 40790 2 0x00000080 [Tue Mar 17 18:01:31 2020][8592576.236767] Call Trace: [Tue Mar 17 18:01:31 2020][8592576.239398] [] schedule+0x29/0x70 [Tue Mar 17 18:01:31 2020][8592576.244547] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.251599] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:31 2020][8592576.257609] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.264852] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.271566] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.279307] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:31 2020][8592576.285497] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:31 2020][8592576.292295] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.299611] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.307184] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:31 2020][8592576.314324] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:31 2020][8592576.320598] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:01:31 2020][8592576.327148] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:31 2020][8592576.333455] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.340533] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.348281] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:31 2020][8592576.355533] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.363392] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.370359] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:31 2020][8592576.375788] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.382260] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:31 2020][8592576.389832] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:31 2020][8592576.394902] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592576.401172] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:31 2020][8592576.407791] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:31 2020][8592576.414056] ll_ost_io03_082 D ffff9c4b973a4100 0 40791 2 0x00000080 [Tue Mar 17 18:01:31 2020][8592576.421340] Call Trace: [Tue Mar 17 18:01:31 2020][8592576.423980] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.431642] [] schedule+0x29/0x70 [Tue Mar 17 18:01:32 2020][8592576.436785] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.443837] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:32 2020][8592576.449868] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.457103] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.463810] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.471553] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:32 2020][8592576.477742] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.484540] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.491872] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.499447] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.506588] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:32 2020][8592576.512862] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:32 2020][8592576.519655] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:32 2020][8592576.525888] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.532594] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:32 2020][8592576.538346] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:32 2020][8592576.543865] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.550945] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.558692] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:32 2020][8592576.565943] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.573806] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.580772] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:32 2020][8592576.586208] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.592697] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.600270] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:32 2020][8592576.605323] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592576.611588] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:32 2020][8592576.618202] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592576.624477] ll_ost_io00_092 D ffff9c4b973a6180 0 40792 2 0x00000080 [Tue Mar 17 18:01:32 2020][8592576.631744] Call Trace: [Tue Mar 17 18:01:32 2020][8592576.634375] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:32 2020][8592576.640996] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:32 2020][8592576.647357] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:32 2020][8592576.652692] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:32 2020][8592576.658187] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.665158] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.672552] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:32 2020][8592576.678572] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:32 2020][8592576.684930] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:32 2020][8592576.691548] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:32 2020][8592576.697034] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:32 2020][8592576.702259] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:32 2020][8592576.709393] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:32 2020][8592576.714643] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:32 2020][8592576.720309] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.727709] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.734680] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.741994] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.748262] LustreError: dumping log to /tmp/lustre-log.1584493292.73710 [Tue Mar 17 18:01:32 2020][8592576.755756] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.763085] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:32 2020][8592576.769970] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:32 2020][8592576.776186] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.782930] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.789574] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.796644] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:32 2020][8592576.803955] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:32 2020][8592576.809701] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:32 2020][8592576.815221] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.822306] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.830057] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:32 2020][8592576.837309] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.845168] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.852150] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:32 2020][8592576.857579] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.864054] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592576.871624] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:32 2020][8592576.876680] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592576.882952] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:32 2020][8592576.889566] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592576.895840] ll_ost_io00_093 D ffff9c4b973a1040 0 40793 2 0x00000080 [Tue Mar 17 18:01:32 2020][8592576.903107] Call Trace: [Tue Mar 17 18:01:32 2020][8592576.905739] [] schedule+0x29/0x70 [Tue Mar 17 18:01:32 2020][8592576.910889] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.917956] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:32 2020][8592576.923966] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.931198] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.937906] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.945649] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:32 2020][8592576.951839] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592576.958637] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.965951] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.973527] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592576.980681] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:32 2020][8592576.986956] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:32 2020][8592576.993753] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:32 2020][8592576.999969] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.006671] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:32 2020][8592577.012712] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.020145] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.027227] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.034978] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:32 2020][8592577.042235] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.050091] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.057075] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:32 2020][8592577.062505] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.068980] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.076551] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:32 2020][8592577.081613] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592577.087887] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:32 2020][8592577.094508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592577.100784] ll_ost_io00_094 D ffff9c5154108000 0 40794 2 0x00000080 [Tue Mar 17 18:01:32 2020][8592577.108058] Call Trace: [Tue Mar 17 18:01:32 2020][8592577.110689] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:32 2020][8592577.117303] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:32 2020][8592577.123678] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:32 2020][8592577.128994] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:32 2020][8592577.134496] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.141470] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.148865] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:32 2020][8592577.154878] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:32 2020][8592577.161239] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:32 2020][8592577.167859] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:32 2020][8592577.174239] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:32 2020][8592577.181373] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:32 2020][8592577.186599] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:32 2020][8592577.192267] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.199666] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592577.206634] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.213949] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.220832] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.228145] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:32 2020][8592577.235027] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:32 2020][8592577.241267] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.247999] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.254649] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.261700] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:32 2020][8592577.267225] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.274304] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.282052] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:32 2020][8592577.289305] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.297165] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.304131] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:32 2020][8592577.309566] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.316054] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:32 2020][8592577.323621] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:32 2020][8592577.328682] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592577.334948] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:32 2020][8592577.341562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:32 2020][8592577.347828] ll_ost_io02_100 D ffff9c6ae992b0c0 0 40795 2 0x00000080 [Tue Mar 17 18:01:32 2020][8592577.355103] Call Trace: [Tue Mar 17 18:01:32 2020][8592577.357742] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.365397] [] schedule+0x29/0x70 [Tue Mar 17 18:01:32 2020][8592577.370581] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:32 2020][8592577.377631] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:32 2020][8592577.383640] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:32 2020][8592577.390868] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:32 2020][8592577.397580] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:32 2020][8592577.405325] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:33 2020][8592577.411514] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.418311] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.425643] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.433219] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.440373] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:33 2020][8592577.446651] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:33 2020][8592577.453445] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:33 2020][8592577.459663] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.466400] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.473048] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.480099] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:33 2020][8592577.485617] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.492696] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.500447] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:33 2020][8592577.507712] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.515576] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.522540] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:33 2020][8592577.527969] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.534442] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.542015] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:33 2020][8592577.547076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592577.553352] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:33 2020][8592577.559973] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592577.566245] ll_ost_io03_083 D ffff9c6ae98ea080 0 40796 2 0x00000080 [Tue Mar 17 18:01:33 2020][8592577.573533] Call Trace: [Tue Mar 17 18:01:33 2020][8592577.576162] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:33 2020][8592577.581483] [] schedule+0x29/0x70 [Tue Mar 17 18:01:33 2020][8592577.586621] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:33 2020][8592577.592630] [] ? __radix_tree_lookup+0x84/0xf0 [Tue Mar 17 18:01:33 2020][8592577.598901] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:33 2020][8592577.604215] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:33 2020][8592577.610396] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:33 2020][8592577.615803] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:33 2020][8592577.621202] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:33 2020][8592577.627221] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:33 2020][8592577.632537] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:33 2020][8592577.639411] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592577.645505] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:33 2020][8592577.651089] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.657973] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.664764] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.672344] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.680613] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.688454] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.695107] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:01:33 2020][8592577.701933] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.709888] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.717054] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.723774] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:01:33 2020][8592577.730574] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.737628] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.744781] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:33 2020][8592577.752092] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:33 2020][8592577.758279] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:33 2020][8592577.765074] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:33 2020][8592577.771303] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.772249] LustreError: dumping log to /tmp/lustre-log.1584493293.68747 [Tue Mar 17 18:01:33 2020][8592577.784911] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.791561] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.798627] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:33 2020][8592577.805934] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:33 2020][8592577.811689] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:33 2020][8592577.817214] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.824295] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.832059] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:33 2020][8592577.839315] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.847174] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.854138] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:33 2020][8592577.859567] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.866040] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592577.873613] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:33 2020][8592577.878673] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592577.884938] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:33 2020][8592577.891552] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592577.897841] ll_ost_io03_084 D ffff9c7a20e85140 0 40797 2 0x00000080 [Tue Mar 17 18:01:33 2020][8592577.905113] Call Trace: [Tue Mar 17 18:01:33 2020][8592577.907741] [] schedule+0x29/0x70 [Tue Mar 17 18:01:33 2020][8592577.912893] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.919944] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:33 2020][8592577.925960] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.933188] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:33 2020][8592577.939031] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.945744] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.953052] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:33 2020][8592577.959258] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:33 2020][8592577.966057] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.973374] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.980948] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592577.988086] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:33 2020][8592577.994880] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:33 2020][8592578.001099] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.007828] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.014477] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.021528] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:33 2020][8592578.027581] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.035012] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.042089] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.049836] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:33 2020][8592578.057090] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.064951] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.071916] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:33 2020][8592578.077354] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.083828] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.091398] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:33 2020][8592578.096452] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592578.102744] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:33 2020][8592578.109365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592578.115639] ll_ost_io02_101 D ffff9c7a32a88000 0 40798 2 0x00000080 [Tue Mar 17 18:01:33 2020][8592578.122907] Call Trace: [Tue Mar 17 18:01:33 2020][8592578.125538] [] schedule+0x29/0x70 [Tue Mar 17 18:01:33 2020][8592578.130686] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:33 2020][8592578.137739] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:33 2020][8592578.143757] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:33 2020][8592578.150984] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:33 2020][8592578.157695] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.165438] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:33 2020][8592578.171620] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:33 2020][8592578.178417] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.185733] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.193307] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.200447] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:33 2020][8592578.206723] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:33 2020][8592578.213518] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:33 2020][8592578.219733] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.226479] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.233129] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.240196] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:33 2020][8592578.247502] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:33 2020][8592578.253022] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.260102] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.267849] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:33 2020][8592578.275102] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.282961] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.289928] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:33 2020][8592578.295379] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.301856] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:33 2020][8592578.309427] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:33 2020][8592578.314480] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592578.320755] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:33 2020][8592578.327375] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:33 2020][8592578.333641] ll_ost_io01_100 D ffff9c7a2e520000 0 40799 2 0x00000080 [Tue Mar 17 18:01:33 2020][8592578.340911] Call Trace: [Tue Mar 17 18:01:33 2020][8592578.343540] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:33 2020][8592578.350161] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:33 2020][8592578.356557] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:33 2020][8592578.361879] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:33 2020][8592578.367381] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.374365] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.381760] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:33 2020][8592578.387781] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:33 2020][8592578.394141] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:33 2020][8592578.400764] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:33 2020][8592578.407468] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:33 2020][8592578.414609] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:33 2020][8592578.419856] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:33 2020][8592578.425538] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.432940] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.439909] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.447226] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.454113] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.461427] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:34 2020][8592578.468310] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:34 2020][8592578.474541] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.481274] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.487925] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.495006] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:34 2020][8592578.502314] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:34 2020][8592578.508068] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:34 2020][8592578.514619] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.522045] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.529126] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.536872] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:34 2020][8592578.544127] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.551984] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.558950] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:34 2020][8592578.564394] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.570871] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.578443] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:34 2020][8592578.583502] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592578.589771] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:34 2020][8592578.596389] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592578.602655] ll_ost_io03_085 D ffff9c714edd8000 0 40800 2 0x00000080 [Tue Mar 17 18:01:34 2020][8592578.609924] Call Trace: [Tue Mar 17 18:01:34 2020][8592578.612556] [] schedule+0x29/0x70 [Tue Mar 17 18:01:34 2020][8592578.617707] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.624773] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:34 2020][8592578.630783] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.638008] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:34 2020][8592578.643845] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.650558] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.657865] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:34 2020][8592578.664056] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.670852] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.678168] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.685756] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.692899] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:34 2020][8592578.699694] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:34 2020][8592578.705909] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.712642] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.719289] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.726340] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:34 2020][8592578.732374] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.739807] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.746885] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.754647] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:34 2020][8592578.761904] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.769763] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.776729] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:34 2020][8592578.782166] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.788641] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.796211] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:34 2020][8592578.796259] LustreError: dumping log to /tmp/lustre-log.1584493294.10668 [Tue Mar 17 18:01:34 2020][8592578.808152] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592578.814421] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:34 2020][8592578.821040] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592578.827306] ll_ost_io02_102 D ffff9c714eddd140 0 40801 2 0x00000080 [Tue Mar 17 18:01:34 2020][8592578.834576] Call Trace: [Tue Mar 17 18:01:34 2020][8592578.837205] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:34 2020][8592578.843220] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:34 2020][8592578.849842] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:34 2020][8592578.856201] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:34 2020][8592578.861514] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:34 2020][8592578.867008] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.873975] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.881372] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:34 2020][8592578.887650] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:34 2020][8592578.894015] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.901245] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.908907] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:34 2020][8592578.914492] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.922167] [] ? qsd_acquire+0x19b/0xcb0 [lquota] [Tue Mar 17 18:01:34 2020][8592578.928708] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.936114] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.943174] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.951360] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592578.958345] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.965574] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592578.972887] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:34 2020][8592578.979681] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:34 2020][8592578.985898] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.992628] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592578.999277] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.006328] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:34 2020][8592579.012368] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.019809] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.026890] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.034637] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:34 2020][8592579.041889] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.049752] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.056716] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:34 2020][8592579.062153] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.068626] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.076199] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:34 2020][8592579.081260] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592579.087549] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:34 2020][8592579.094164] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592579.100430] ll_ost_io00_095 D ffff9c714edda080 0 40802 2 0x00000080 [Tue Mar 17 18:01:34 2020][8592579.107700] Call Trace: [Tue Mar 17 18:01:34 2020][8592579.110329] [] schedule+0x29/0x70 [Tue Mar 17 18:01:34 2020][8592579.115479] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.122530] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:34 2020][8592579.128547] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.135774] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:34 2020][8592579.141617] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.148347] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.155654] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:34 2020][8592579.161836] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.168636] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.175951] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.183524] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.190665] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:34 2020][8592579.197459] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:34 2020][8592579.203677] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.210407] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.217056] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.224120] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:34 2020][8592579.229641] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.236720] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.244469] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:34 2020][8592579.251722] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.259582] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.266547] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:34 2020][8592579.271983] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.278457] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.286028] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:34 2020][8592579.291096] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592579.297366] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:34 2020][8592579.303985] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:34 2020][8592579.310252] ll_ost_io02_103 D ffff9c714eddc100 0 40803 2 0x00000080 [Tue Mar 17 18:01:34 2020][8592579.317520] Call Trace: [Tue Mar 17 18:01:34 2020][8592579.320157] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.327812] [] schedule+0x29/0x70 [Tue Mar 17 18:01:34 2020][8592579.332964] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.340014] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:34 2020][8592579.346024] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.353265] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.359992] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.367738] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:34 2020][8592579.373938] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:34 2020][8592579.380735] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.388050] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.395627] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:34 2020][8592579.402766] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:34 2020][8592579.409042] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:34 2020][8592579.415850] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:34 2020][8592579.422071] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:34 2020][8592579.428823] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.435474] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.442557] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:35 2020][8592579.449864] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:35 2020][8592579.455611] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:35 2020][8592579.462171] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.469595] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.476677] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.484438] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:35 2020][8592579.491693] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.499553] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.506518] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:35 2020][8592579.511946] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.518420] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.525992] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:35 2020][8592579.531046] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592579.537322] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:35 2020][8592579.543943] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592579.550218] ll_ost_io01_101 D ffff9c714eddb0c0 0 40804 2 0x00000080 [Tue Mar 17 18:01:35 2020][8592579.557510] Call Trace: [Tue Mar 17 18:01:35 2020][8592579.560140] [] schedule+0x29/0x70 [Tue Mar 17 18:01:35 2020][8592579.565286] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.572340] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.579475] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:35 2020][8592579.585485] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:35 2020][8592579.591675] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:35 2020][8592579.598468] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:35 2020][8592579.604684] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.611429] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.618073] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.625125] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:35 2020][8592579.630650] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.637729] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.645476] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:35 2020][8592579.652730] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.660590] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.667557] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:35 2020][8592579.672992] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.679479] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592579.687047] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:35 2020][8592579.692108] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592579.698384] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:35 2020][8592579.705003] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592579.711270] ll_ost_io03_086 D ffff9c6e21979040 0 40805 2 0x00000080 [Tue Mar 17 18:01:35 2020][8592579.718540] Call Trace: [Tue Mar 17 18:01:35 2020][8592579.721167] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:35 2020][8592579.726740] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:35 2020][8592579.732052] [] schedule+0x29/0x70 [Tue Mar 17 18:01:35 2020][8592579.737208] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:35 2020][8592579.743224] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.750879] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:35 2020][8592579.756460] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:35 2020][8592579.762300] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:35 2020][8592579.767611] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:35 2020][8592579.773792] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:35 2020][8592579.779192] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:35 2020][8592579.784601] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:35 2020][8592579.790613] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:35 2020][8592579.795942] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:35 2020][8592579.802818] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592579.808919] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:35 2020][8592579.814502] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:35 2020][8592579.820252] LustreError: dumping log to /tmp/lustre-log.1584493295.40838 [Tue Mar 17 18:01:35 2020][8592579.828260] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.835066] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:35 2020][8592579.842639] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.850906] [] ldiskfs_quota_write+0xc6/0x1e0 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.857956] [] qtree_write_dquot+0xd5/0x180 [Tue Mar 17 18:01:35 2020][8592579.863970] [] v2_write_dquot+0x2b/0x30 [Tue Mar 17 18:01:35 2020][8592579.869637] [] dquot_commit+0xbd/0xd0 [Tue Mar 17 18:01:35 2020][8592579.875131] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.882101] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.889501] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:35 2020][8592579.895775] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.902656] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.911011] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.918245] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.925903] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:35 2020][8592579.931503] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.939170] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.947092] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.954491] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.961541] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:35 2020][8592579.967042] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.975222] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.982449] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592579.989762] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:35 2020][8592579.996556] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:35 2020][8592580.002791] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.009523] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.016171] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.023222] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:35 2020][8592580.028747] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.035827] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.043574] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:35 2020][8592580.050829] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.058691] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.065653] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:35 2020][8592580.071092] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.077579] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.085152] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:35 2020][8592580.090205] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592580.096472] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:35 2020][8592580.103092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592580.109357] ll_ost_io00_096 D ffff9c7a30dc8000 0 40806 2 0x00000080 [Tue Mar 17 18:01:35 2020][8592580.116628] Call Trace: [Tue Mar 17 18:01:35 2020][8592580.119257] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:35 2020][8592580.125879] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:35 2020][8592580.132256] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:35 2020][8592580.137569] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:35 2020][8592580.143071] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.150037] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.157434] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:35 2020][8592580.163451] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:35 2020][8592580.169814] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:35 2020][8592580.176431] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:35 2020][8592580.181917] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:35 2020][8592580.187155] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:35 2020][8592580.194309] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:35 2020][8592580.199536] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:35 2020][8592580.205200] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.212601] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:35 2020][8592580.219570] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.226886] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.233766] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.241080] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:35 2020][8592580.247962] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:35 2020][8592580.254178] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.260921] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.268348] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.275427] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.283176] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:35 2020][8592580.290427] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.298275] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 18:01:35 2020][8592580.305611] [] ? ptlrpc_at_add_timed+0xe5/0x230 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.312779] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.319255] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:35 2020][8592580.326823] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:35 2020][8592580.331893] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592580.338160] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:35 2020][8592580.344775] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:35 2020][8592580.351048] ll_ost_io02_104 D ffff9c6a96cc30c0 0 40807 2 0x00000080 [Tue Mar 17 18:01:35 2020][8592580.358316] Call Trace: [Tue Mar 17 18:01:35 2020][8592580.360946] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:35 2020][8592580.367566] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:35 2020][8592580.373921] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:35 2020][8592580.379244] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:35 2020][8592580.384741] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.391727] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.399123] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:35 2020][8592580.405404] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.412285] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.420640] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:35 2020][8592580.427869] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.435539] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:36 2020][8592580.441120] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.448781] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.456702] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.464103] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.471176] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.479353] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:36 2020][8592580.486326] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.493552] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.500866] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:36 2020][8592580.507659] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:36 2020][8592580.513876] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.520606] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.527260] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.534330] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:36 2020][8592580.539852] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.546939] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.554688] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:36 2020][8592580.561940] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.569799] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.576765] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:36 2020][8592580.582200] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.588676] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.596247] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:36 2020][8592580.601314] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592580.607582] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:36 2020][8592580.614195] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592580.620462] ll_ost_io01_102 D ffff9c6a96cc2080 0 40808 2 0x00000080 [Tue Mar 17 18:01:36 2020][8592580.627729] Call Trace: [Tue Mar 17 18:01:36 2020][8592580.630361] [] schedule+0x29/0x70 [Tue Mar 17 18:01:36 2020][8592580.635514] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.642569] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.649705] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:36 2020][8592580.655722] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:36 2020][8592580.661924] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:36 2020][8592580.668714] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:36 2020][8592580.674929] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.681633] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:36 2020][8592580.687589] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.694238] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.701285] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:36 2020][8592580.707322] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.714755] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.721834] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.729594] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:36 2020][8592580.736850] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.744712] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.751677] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:36 2020][8592580.757113] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.763587] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.771159] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:36 2020][8592580.776211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592580.782478] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:36 2020][8592580.789090] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592580.795369] ll_ost_io02_105 D ffff9c6a96cc1040 0 40809 2 0x00000080 [Tue Mar 17 18:01:36 2020][8592580.802644] Call Trace: [Tue Mar 17 18:01:36 2020][8592580.805279] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.812942] [] schedule+0x29/0x70 [Tue Mar 17 18:01:36 2020][8592580.818093] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:36 2020][8592580.825146] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:36 2020][8592580.831162] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:36 2020][8592580.838391] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:36 2020][8592580.844267] LustreError: dumping log to /tmp/lustre-log.1584493296.68714 [Tue Mar 17 18:01:36 2020][8592580.851982] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.859740] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:36 2020][8592580.865923] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:36 2020][8592580.872721] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.880036] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.887612] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592580.894751] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:36 2020][8592580.901027] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:36 2020][8592580.907821] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:36 2020][8592580.914036] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.920769] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.927415] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.934499] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:36 2020][8592580.941808] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:36 2020][8592580.947564] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:01:36 2020][8592580.953230] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:36 2020][8592580.958756] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.965837] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.973584] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:36 2020][8592580.980839] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.988697] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592580.995679] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:36 2020][8592581.001454] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.007927] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.015499] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:36 2020][8592581.020554] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592581.026828] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:36 2020][8592581.033449] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592581.039715] ll_ost_io03_087 D ffff9c6a96cc5140 0 40810 2 0x00000080 [Tue Mar 17 18:01:36 2020][8592581.046984] Call Trace: [Tue Mar 17 18:01:36 2020][8592581.049613] [] schedule+0x29/0x70 [Tue Mar 17 18:01:36 2020][8592581.054764] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.061830] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:36 2020][8592581.067841] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.075068] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:36 2020][8592581.080912] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.087628] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.094940] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:36 2020][8592581.101122] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.107921] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.115238] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.122830] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.129976] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:36 2020][8592581.136768] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:36 2020][8592581.142985] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.149717] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.156335] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:36 2020][8592581.162090] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:36 2020][8592581.168104] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 18:01:36 2020][8592581.174137] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.181572] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.188664] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.196415] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:36 2020][8592581.203666] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.211528] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.218492] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:36 2020][8592581.223919] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.230394] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:36 2020][8592581.237969] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:36 2020][8592581.243028] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592581.249301] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:36 2020][8592581.255930] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592581.262197] ll_ost_io01_103 D ffff9c6a96cc0000 0 40811 2 0x00000080 [Tue Mar 17 18:01:36 2020][8592581.269467] Call Trace: [Tue Mar 17 18:01:36 2020][8592581.272096] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:36 2020][8592581.277417] [] schedule+0x29/0x70 [Tue Mar 17 18:01:36 2020][8592581.282557] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:36 2020][8592581.288575] [] ? physflat_send_IPI_mask+0xe/0x10 [Tue Mar 17 18:01:36 2020][8592581.295021] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:36 2020][8592581.300861] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:36 2020][8592581.306173] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:36 2020][8592581.312368] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:36 2020][8592581.317769] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:36 2020][8592581.323168] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:36 2020][8592581.328744] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:36 2020][8592581.334062] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:36 2020][8592581.340502] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:36 2020][8592581.346599] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.353482] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.356249] LNet: Service thread pid 68890 was inactive for 200.49s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:36 2020][8592581.356250] LNet: Skipped 163 previous similar messages [Tue Mar 17 18:01:36 2020][8592581.378727] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:36 2020][8592581.386306] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.394574] [] osd_ldiskfs_write_record+0xd9/0x410 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.402417] [] osd_write+0x140/0x5c0 [osd_ldiskfs] [Tue Mar 17 18:01:36 2020][8592581.409050] [] dt_record_write+0x36/0x120 [obdclass] [Tue Mar 17 18:01:37 2020][8592581.415876] [] tgt_client_data_write.isra.18+0x167/0x180 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.423824] [] tgt_last_rcvd_update+0x656/0xc90 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.431004] [] tgt_txn_stop_cb+0x19d/0x490 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.437712] [] ? generic_setxattr+0x68/0x80 [Tue Mar 17 18:01:37 2020][8592581.443757] [] dt_txn_hook_stop+0x81/0xd0 [obdclass] [Tue Mar 17 18:01:37 2020][8592581.450567] [] osd_trans_stop+0x18d/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.457636] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.464779] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:37 2020][8592581.470967] [] ofd_object_punch+0x4bb/0xd30 [ofd] [Tue Mar 17 18:01:37 2020][8592581.477516] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:37 2020][8592581.483823] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.490902] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.498650] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:37 2020][8592581.505905] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.513763] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.520728] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:37 2020][8592581.526179] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.532656] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.540228] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:37 2020][8592581.545288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592581.551556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:37 2020][8592581.558176] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592581.564441] ll_ost_io01_104 D ffff9c75fcd78000 0 40812 2 0x00000080 [Tue Mar 17 18:01:37 2020][8592581.571709] Call Trace: [Tue Mar 17 18:01:37 2020][8592581.574341] [] schedule+0x29/0x70 [Tue Mar 17 18:01:37 2020][8592581.579493] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:37 2020][8592581.586558] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:37 2020][8592581.592569] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592581.599795] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:37 2020][8592581.605638] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:37 2020][8592581.612353] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.619668] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:37 2020][8592581.625857] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592581.632654] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.639971] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.647559] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.654700] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:37 2020][8592581.661496] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:37 2020][8592581.667713] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.674443] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.681091] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.688144] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:37 2020][8592581.693668] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.700748] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.708499] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:37 2020][8592581.715763] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.723626] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.730592] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:37 2020][8592581.736030] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.742504] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.750074] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:37 2020][8592581.755127] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592581.761394] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:37 2020][8592581.768015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592581.774289] ll_ost_io00_097 D ffff9c75fcd79040 0 40813 2 0x00000080 [Tue Mar 17 18:01:37 2020][8592581.781559] Call Trace: [Tue Mar 17 18:01:37 2020][8592581.784185] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:37 2020][8592581.790814] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:37 2020][8592581.797169] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:37 2020][8592581.802484] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:37 2020][8592581.807985] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.814951] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.822348] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:37 2020][8592581.828630] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.835519] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.843887] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.851119] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.858775] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:37 2020][8592581.864362] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.868279] LustreError: dumping log to /tmp/lustre-log.1584493297.68709 [Tue Mar 17 18:01:37 2020][8592581.878908] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.886836] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.894248] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.901310] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.909496] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592581.916481] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.923711] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592581.931022] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:37 2020][8592581.937817] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:37 2020][8592581.944034] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.950765] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.957414] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.964481] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:37 2020][8592581.971788] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:37 2020][8592581.977542] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:37 2020][8592581.984121] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.991554] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592581.998633] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.006380] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:37 2020][8592582.013635] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.021493] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.028460] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:37 2020][8592582.033898] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.040370] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.047941] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:37 2020][8592582.053008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592582.059278] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:37 2020][8592582.065898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592582.072167] ll_ost_io03_088 D ffff9c75fcd7a080 0 40814 2 0x00000080 [Tue Mar 17 18:01:37 2020][8592582.079443] Call Trace: [Tue Mar 17 18:01:37 2020][8592582.082072] [] schedule+0x29/0x70 [Tue Mar 17 18:01:37 2020][8592582.087223] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.094276] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:37 2020][8592582.100291] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.107518] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:37 2020][8592582.113360] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.120089] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.127399] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:37 2020][8592582.133587] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.140387] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.147704] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.155275] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.162416] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:37 2020][8592582.169210] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:37 2020][8592582.175426] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.182173] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.188825] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.195873] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:37 2020][8592582.201395] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.208473] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.216218] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:37 2020][8592582.223471] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.231333] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.238299] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:37 2020][8592582.243735] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.250223] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.257789] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:37 2020][8592582.262849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592582.269117] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:37 2020][8592582.275730] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:37 2020][8592582.282002] ll_ost_io02_106 D ffff9c75fcd7b0c0 0 40815 2 0x00000080 [Tue Mar 17 18:01:37 2020][8592582.289272] Call Trace: [Tue Mar 17 18:01:37 2020][8592582.291904] [] schedule+0x29/0x70 [Tue Mar 17 18:01:37 2020][8592582.297054] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.304102] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:37 2020][8592582.310114] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.317351] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:37 2020][8592582.323191] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.329905] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.337210] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:37 2020][8592582.343394] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:37 2020][8592582.350191] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.357507] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.365102] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:37 2020][8592582.372252] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:37 2020][8592582.379048] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:37 2020][8592582.385286] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.392023] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.398670] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.405722] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:37 2020][8592582.411761] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.419186] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.426282] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:37 2020][8592582.434045] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:38 2020][8592582.441302] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.449161] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.456154] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:38 2020][8592582.461589] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.468063] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.475636] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:38 2020][8592582.480699] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.486973] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:38 2020][8592582.493592] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.499860] ll_ost_io01_105 D ffff9c75fcd7e180 0 40816 2 0x00000080 [Tue Mar 17 18:01:38 2020][8592582.507138] Call Trace: [Tue Mar 17 18:01:38 2020][8592582.509765] [] schedule+0x29/0x70 [Tue Mar 17 18:01:38 2020][8592582.514908] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:38 2020][8592582.521975] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:38 2020][8592582.527984] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592582.535213] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:38 2020][8592582.541056] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:38 2020][8592582.547767] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.555075] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:38 2020][8592582.561267] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592582.568064] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.575395] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.582971] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.590110] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:38 2020][8592582.596904] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:38 2020][8592582.603120] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.609824] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:38 2020][8592582.615350] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.622430] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.630176] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:38 2020][8592582.637430] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.645304] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.652264] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:38 2020][8592582.657702] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.664175] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.671747] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:38 2020][8592582.676807] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.683076] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:38 2020][8592582.689696] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.695969] ll_ost_io01_106 D ffff9c75fcd7d140 0 40817 2 0x00000080 [Tue Mar 17 18:01:38 2020][8592582.703239] Call Trace: [Tue Mar 17 18:01:38 2020][8592582.705870] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:38 2020][8592582.711189] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:38 2020][8592582.717825] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:38 2020][8592582.724181] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:38 2020][8592582.729504] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:38 2020][8592582.735002] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.741971] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.749367] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:38 2020][8592582.755649] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.762537] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.770908] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.778137] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.785793] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:38 2020][8592582.791382] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.799041] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.806963] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.814364] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.821420] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.829599] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592582.836569] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.843796] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592582.851124] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:38 2020][8592582.857919] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:38 2020][8592582.864139] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.870870] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.877517] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.884567] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:38 2020][8592582.890607] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.892269] LustreError: dumping log to /tmp/lustre-log.1584493298.68754 [Tue Mar 17 18:01:38 2020][8592582.904922] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.912001] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.919750] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:38 2020][8592582.927002] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.934865] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.941827] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:38 2020][8592582.947260] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.953737] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592582.961311] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:38 2020][8592582.966373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.972647] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:38 2020][8592582.979258] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592582.985540] ll_ost_io03_089 D ffff9c8a2fd09040 0 40818 2 0x00000080 [Tue Mar 17 18:01:38 2020][8592582.992810] Call Trace: [Tue Mar 17 18:01:38 2020][8592582.995441] [] schedule+0x29/0x70 [Tue Mar 17 18:01:38 2020][8592583.000592] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.007642] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:38 2020][8592583.013652] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.020880] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.027593] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.035336] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:38 2020][8592583.041538] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.048332] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.055647] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.063220] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.070360] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:38 2020][8592583.076636] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:38 2020][8592583.083431] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:38 2020][8592583.089645] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.096377] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.103024] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.110090] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:38 2020][8592583.115611] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.122690] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.130438] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:38 2020][8592583.137690] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.145553] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.152516] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:38 2020][8592583.157952] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.164428] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:38 2020][8592583.172000] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:38 2020][8592583.177061] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592583.183350] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:38 2020][8592583.189966] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592583.196240] ll_ost_io02_107 D ffff9c8a2fd08000 0 40819 2 0x00000080 [Tue Mar 17 18:01:38 2020][8592583.203517] Call Trace: [Tue Mar 17 18:01:38 2020][8592583.206145] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:38 2020][8592583.211467] [] schedule+0x29/0x70 [Tue Mar 17 18:01:38 2020][8592583.216606] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:38 2020][8592583.222613] [] ? ___slab_alloc+0x209/0x4f0 [Tue Mar 17 18:01:38 2020][8592583.228540] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:38 2020][8592583.233869] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:38 2020][8592583.240050] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:38 2020][8592583.245459] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:38 2020][8592583.250865] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:38 2020][8592583.256440] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:38 2020][8592583.261760] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:38 2020][8592583.268200] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:38 2020][8592583.274307] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.281188] [] ? ldiskfs_getblk+0xa6/0x200 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.287981] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.295572] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.303845] [] ldiskfs_quota_write+0xc6/0x1e0 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.310893] [] qtree_write_dquot+0xd5/0x180 [Tue Mar 17 18:01:38 2020][8592583.316907] [] v2_write_dquot+0x2b/0x30 [Tue Mar 17 18:01:38 2020][8592583.322566] [] dquot_commit+0xbd/0xd0 [Tue Mar 17 18:01:38 2020][8592583.328068] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.335036] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.342431] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:38 2020][8592583.348452] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:38 2020][8592583.354813] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:38 2020][8592583.361449] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:38 2020][8592583.368602] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:38 2020][8592583.373829] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:38 2020][8592583.379496] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.386893] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:38 2020][8592583.393863] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.401179] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:38 2020][8592583.408062] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.415373] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:39 2020][8592583.422256] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:39 2020][8592583.428488] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.435206] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:39 2020][8592583.441240] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.448685] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.455770] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.463519] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:39 2020][8592583.470772] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.478632] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.485595] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:39 2020][8592583.491024] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.497515] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.505088] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:39 2020][8592583.510149] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.516422] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:39 2020][8592583.523037] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.529309] ll_ost_io01_107 D ffff9c8a04e90000 0 40820 2 0x00000080 [Tue Mar 17 18:01:39 2020][8592583.536580] Call Trace: [Tue Mar 17 18:01:39 2020][8592583.539210] [] schedule+0x29/0x70 [Tue Mar 17 18:01:39 2020][8592583.544359] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.551410] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:39 2020][8592583.557420] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.564661] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:39 2020][8592583.570499] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.577211] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.584518] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:39 2020][8592583.590709] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.597507] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.604824] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.612403] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.619544] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:39 2020][8592583.626338] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:39 2020][8592583.632572] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.639304] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.645951] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.653002] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:39 2020][8592583.658531] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.665609] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.673355] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:39 2020][8592583.680609] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.688470] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.695435] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:39 2020][8592583.700871] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.707346] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.714919] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:39 2020][8592583.719979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.726253] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:39 2020][8592583.732865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.739133] ll_ost_io00_098 D ffff9c8a30bb6180 0 40821 2 0x00000080 [Tue Mar 17 18:01:39 2020][8592583.746408] Call Trace: [Tue Mar 17 18:01:39 2020][8592583.749036] [] schedule+0x29/0x70 [Tue Mar 17 18:01:39 2020][8592583.754182] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.761253] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:39 2020][8592583.767269] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.774494] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:39 2020][8592583.780337] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.787052] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.794365] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:39 2020][8592583.800547] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592583.807346] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.814661] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.822255] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.829391] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:39 2020][8592583.836186] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:39 2020][8592583.842402] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.849133] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.855783] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.862831] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:39 2020][8592583.868352] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.875430] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.883178] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:39 2020][8592583.890445] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.898300] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.905264] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:39 2020][8592583.910694] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.916273] LustreError: dumping log to /tmp/lustre-log.1584493299.68885 [Tue Mar 17 18:01:39 2020][8592583.924040] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592583.931612] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:39 2020][8592583.936674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.942947] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:39 2020][8592583.949562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592583.955850] ll_ost_io03_090 D ffff9c8a30bb0000 0 40822 2 0x00000080 [Tue Mar 17 18:01:39 2020][8592583.963120] Call Trace: [Tue Mar 17 18:01:39 2020][8592583.965751] [] schedule+0x29/0x70 [Tue Mar 17 18:01:39 2020][8592583.970904] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.977960] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:39 2020][8592583.985096] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:39 2020][8592583.991112] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:39 2020][8592583.997300] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:39 2020][8592584.004096] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:39 2020][8592584.010311] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.017057] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.023708] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.030781] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:39 2020][8592584.038092] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:39 2020][8592584.043846] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:39 2020][8592584.049365] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.056442] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.064190] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:39 2020][8592584.071444] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.079304] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.086286] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:39 2020][8592584.091714] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.098191] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.105760] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:39 2020][8592584.110815] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592584.117089] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:39 2020][8592584.123711] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592584.129985] ll_ost_io00_099 D ffff9c5991104100 0 40823 2 0x00000080 [Tue Mar 17 18:01:39 2020][8592584.137261] Call Trace: [Tue Mar 17 18:01:39 2020][8592584.139891] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:39 2020][8592584.146504] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:39 2020][8592584.152880] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:39 2020][8592584.158194] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:39 2020][8592584.163687] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.170655] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.178050] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:39 2020][8592584.184060] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:39 2020][8592584.190424] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:39 2020][8592584.197040] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:39 2020][8592584.204182] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:39 2020][8592584.209415] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:39 2020][8592584.215105] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.222505] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592584.229474] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.236791] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.243672] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:39 2020][8592584.250984] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:39 2020][8592584.257867] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:39 2020][8592584.264083] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.270812] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.277463] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.284546] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:39 2020][8592584.291854] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:39 2020][8592584.297608] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:39 2020][8592584.303134] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.310214] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.317961] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:39 2020][8592584.325216] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.333075] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.340041] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:39 2020][8592584.345492] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.351967] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:39 2020][8592584.359540] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:39 2020][8592584.364616] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592584.370885] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:39 2020][8592584.377506] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:39 2020][8592584.383782] ll_ost_io03_091 D ffff9c59911030c0 0 40824 2 0x00000080 [Tue Mar 17 18:01:39 2020][8592584.391058] Call Trace: [Tue Mar 17 18:01:39 2020][8592584.393686] [] schedule+0x29/0x70 [Tue Mar 17 18:01:39 2020][8592584.398829] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:39 2020][8592584.405897] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:39 2020][8592584.411907] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:39 2020][8592584.419133] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:39 2020][8592584.424978] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:39 2020][8592584.431691] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.439021] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:40 2020][8592584.445205] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.452002] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.459318] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.466892] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.474045] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:40 2020][8592584.480843] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:40 2020][8592584.487059] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.493793] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.500438] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.507490] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:40 2020][8592584.513019] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.520097] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.527844] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:40 2020][8592584.535096] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.542973] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.549941] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:40 2020][8592584.555376] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.561849] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.569422] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:40 2020][8592584.574484] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592584.580758] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:40 2020][8592584.587371] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592584.593646] ll_ost_io01_108 D ffff9c569e9c8000 0 40825 2 0x00000080 [Tue Mar 17 18:01:40 2020][8592584.600921] Call Trace: [Tue Mar 17 18:01:40 2020][8592584.603559] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.611215] [] schedule+0x29/0x70 [Tue Mar 17 18:01:40 2020][8592584.616378] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.623423] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:40 2020][8592584.629433] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.636661] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.643374] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.651114] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:40 2020][8592584.657296] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.664094] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.671410] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.678997] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.686141] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:40 2020][8592584.692414] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:40 2020][8592584.699210] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:40 2020][8592584.705427] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.712130] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:40 2020][8592584.717656] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.724734] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.732483] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:40 2020][8592584.739748] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.747604] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.754570] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:40 2020][8592584.760005] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.766482] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.774052] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:40 2020][8592584.779114] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592584.785389] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:40 2020][8592584.792010] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592584.798285] ll_ost_io01_109 D ffff9c569e9c9040 0 40826 2 0x00000080 [Tue Mar 17 18:01:40 2020][8592584.805570] Call Trace: [Tue Mar 17 18:01:40 2020][8592584.808198] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:40 2020][8592584.814814] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:40 2020][8592584.821180] [] ? __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.829615] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:40 2020][8592584.834936] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:40 2020][8592584.840437] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.847405] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.854800] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:40 2020][8592584.860810] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:40 2020][8592584.867188] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:40 2020][8592584.873807] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:40 2020][8592584.880949] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:40 2020][8592584.886182] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:40 2020][8592584.891846] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.899252] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592584.906223] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.913539] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.920421] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592584.927734] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:40 2020][8592584.934632] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:40 2020][8592584.940258] LustreError: dumping log to /tmp/lustre-log.1584493300.40813 [Tue Mar 17 18:01:40 2020][8592584.947721] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.954426] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:40 2020][8592584.959950] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.967031] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.974779] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:40 2020][8592584.982031] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.989892] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592584.996857] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:40 2020][8592585.002309] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.008786] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.016358] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:40 2020][8592585.021418] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592585.027693] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:40 2020][8592585.034315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592585.040589] ll_ost_io02_108 D ffff9c523155d140 0 40827 2 0x00000080 [Tue Mar 17 18:01:40 2020][8592585.047865] Call Trace: [Tue Mar 17 18:01:40 2020][8592585.050502] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.058157] [] schedule+0x29/0x70 [Tue Mar 17 18:01:40 2020][8592585.063322] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:40 2020][8592585.070376] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:40 2020][8592585.076384] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592585.083613] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:40 2020][8592585.090325] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.098067] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:40 2020][8592585.104260] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592585.111054] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.118371] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.125944] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.133099] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:40 2020][8592585.139376] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:40 2020][8592585.146169] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:40 2020][8592585.152388] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.159119] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.165769] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.172819] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:40 2020][8592585.178343] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.185423] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.193170] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:40 2020][8592585.200425] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.208301] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.215267] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:40 2020][8592585.220696] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.227168] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.234740] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:40 2020][8592585.239793] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592585.246061] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:40 2020][8592585.252681] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:40 2020][8592585.258948] ll_ost_io00_100 D ffff9c523155e180 0 40828 2 0x00000080 [Tue Mar 17 18:01:40 2020][8592585.266240] Call Trace: [Tue Mar 17 18:01:40 2020][8592585.268869] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:40 2020][8592585.275482] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:40 2020][8592585.281835] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:40 2020][8592585.287148] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:40 2020][8592585.292641] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.299612] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.307014] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:40 2020][8592585.313031] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:40 2020][8592585.319394] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:40 2020][8592585.326030] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.332736] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:40 2020][8592585.339879] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:40 2020][8592585.345112] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:40 2020][8592585.350779] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.358176] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:40 2020][8592585.365150] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.372469] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.379350] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:40 2020][8592585.386663] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:40 2020][8592585.393561] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:40 2020][8592585.399782] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.406521] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:40 2020][8592585.413168] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.420218] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:41 2020][8592585.425746] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.432826] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.440589] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:41 2020][8592585.447843] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.452268] LustreError: dumping log to /tmp/lustre-log.1584493301.73703 [Tue Mar 17 18:01:41 2020][8592585.462590] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.469557] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:41 2020][8592585.474986] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.481459] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.489031] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:41 2020][8592585.494086] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.500358] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:41 2020][8592585.506971] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.513238] ll_ost_io03_092 D ffff9c5231559040 0 40829 2 0x00000080 [Tue Mar 17 18:01:41 2020][8592585.520513] Call Trace: [Tue Mar 17 18:01:41 2020][8592585.523151] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.530806] [] schedule+0x29/0x70 [Tue Mar 17 18:01:41 2020][8592585.535970] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.543017] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:41 2020][8592585.549033] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.556263] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.562976] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.570716] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:41 2020][8592585.576907] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.583703] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.591020] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.598608] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.605751] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:41 2020][8592585.612025] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:41 2020][8592585.618819] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:41 2020][8592585.625038] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.631775] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.638422] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.645475] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:41 2020][8592585.651001] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.658096] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.665846] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:41 2020][8592585.673098] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.680959] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.687925] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:41 2020][8592585.693363] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.699834] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.707406] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:41 2020][8592585.712459] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.718725] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:41 2020][8592585.725353] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.731621] ll_ost_io01_110 D ffff9c51be10a080 0 40830 2 0x00000080 [Tue Mar 17 18:01:41 2020][8592585.738890] Call Trace: [Tue Mar 17 18:01:41 2020][8592585.741519] [] schedule+0x29/0x70 [Tue Mar 17 18:01:41 2020][8592585.746662] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.753714] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:41 2020][8592585.759729] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.766957] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:41 2020][8592585.772799] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.779516] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.786829] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:41 2020][8592585.793027] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592585.799825] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.807142] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.814714] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.821857] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:41 2020][8592585.828651] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:41 2020][8592585.834869] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.841606] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.848257] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.855346] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:41 2020][8592585.862656] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:41 2020][8592585.868409] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:41 2020][8592585.873928] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.881007] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.888754] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:41 2020][8592585.896007] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.903867] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.910832] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:41 2020][8592585.916263] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.922748] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592585.930315] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:41 2020][8592585.935368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.941644] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:41 2020][8592585.948265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592585.954539] ll_ost_io01_111 D ffff9c51be10b0c0 0 40831 2 0x00000080 [Tue Mar 17 18:01:41 2020][8592585.961815] Call Trace: [Tue Mar 17 18:01:41 2020][8592585.964445] [] schedule+0x29/0x70 [Tue Mar 17 18:01:41 2020][8592585.969599] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.976654] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592585.983803] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:41 2020][8592585.989818] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:41 2020][8592585.996004] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:41 2020][8592586.002800] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:41 2020][8592586.009017] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.015746] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.022394] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.029447] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:41 2020][8592586.035486] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.042920] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.050000] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.057763] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:41 2020][8592586.065017] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.072877] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.079844] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:41 2020][8592586.085285] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.091755] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.099326] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:41 2020][8592586.104386] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592586.110654] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:41 2020][8592586.117287] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592586.123556] ll_ost_io03_093 D ffff9c51be10c100 0 40832 2 0x00000080 [Tue Mar 17 18:01:41 2020][8592586.130827] Call Trace: [Tue Mar 17 18:01:41 2020][8592586.133464] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.141125] [] schedule+0x29/0x70 [Tue Mar 17 18:01:41 2020][8592586.146276] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.153329] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:41 2020][8592586.159344] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.166574] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.173284] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.181043] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:41 2020][8592586.187226] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.194024] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.201341] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.208911] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.216055] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:41 2020][8592586.222327] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:41 2020][8592586.229121] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:41 2020][8592586.235340] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.242069] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.248732] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.255788] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:41 2020][8592586.261830] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.269264] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.276346] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.284096] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:41 2020][8592586.291349] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.299210] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.306173] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:41 2020][8592586.311602] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.318092] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:41 2020][8592586.325665] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:41 2020][8592586.330718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592586.336993] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:41 2020][8592586.343608] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:41 2020][8592586.349879] ll_ost_io00_101 D ffff9c51be109040 0 40833 2 0x00000080 [Tue Mar 17 18:01:41 2020][8592586.357149] Call Trace: [Tue Mar 17 18:01:41 2020][8592586.359787] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.367448] [] schedule+0x29/0x70 [Tue Mar 17 18:01:41 2020][8592586.372601] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.379667] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:41 2020][8592586.385677] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.392904] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:41 2020][8592586.399615] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:41 2020][8592586.407358] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:41 2020][8592586.413549] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.420345] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.427663] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.435236] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.442391] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:42 2020][8592586.448684] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:42 2020][8592586.455495] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:42 2020][8592586.461715] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.468455] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.475072] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:42 2020][8592586.476265] LustreError: dumping log to /tmp/lustre-log.1584493302.90875 [Tue Mar 17 18:01:42 2020][8592586.487714] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:42 2020][8592586.493722] [] ? enqueue_entity+0x2ef/0xbe0 [Tue Mar 17 18:01:42 2020][8592586.499764] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.507197] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.514279] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.522030] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:42 2020][8592586.529287] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.537144] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.544110] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:42 2020][8592586.549539] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.556014] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.563583] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:42 2020][8592586.568636] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.574904] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:42 2020][8592586.581538] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.587809] ll_ost_io03_094 D ffff9c51be10d140 0 40834 2 0x00000080 [Tue Mar 17 18:01:42 2020][8592586.595084] Call Trace: [Tue Mar 17 18:01:42 2020][8592586.597713] [] schedule+0x29/0x70 [Tue Mar 17 18:01:42 2020][8592586.602858] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.609909] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:42 2020][8592586.615925] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.623152] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:42 2020][8592586.628994] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.635709] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.643023] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:42 2020][8592586.649220] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.656013] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.663329] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.670899] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.678040] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:42 2020][8592586.684835] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:42 2020][8592586.691053] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.697767] [] ? lnet_ptl_attach_md+0x37e/0x530 [lnet] [Tue Mar 17 18:01:42 2020][8592586.704732] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:42 2020][8592586.710273] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.717354] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.725104] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:42 2020][8592586.732355] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.740216] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.747181] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:42 2020][8592586.752617] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.759090] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.766664] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:42 2020][8592586.771725] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.778015] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:42 2020][8592586.784628] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.790896] ll_ost_io02_109 D ffff9c51be10e180 0 40835 2 0x00000080 [Tue Mar 17 18:01:42 2020][8592586.798173] Call Trace: [Tue Mar 17 18:01:42 2020][8592586.800801] [] schedule+0x29/0x70 [Tue Mar 17 18:01:42 2020][8592586.805947] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.813003] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592586.820136] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:42 2020][8592586.826146] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:42 2020][8592586.832335] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:42 2020][8592586.839144] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:42 2020][8592586.845364] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.852094] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.858744] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.865795] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:42 2020][8592586.871321] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.878398] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.886149] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:42 2020][8592586.893400] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.901263] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.908239] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:42 2020][8592586.913671] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.920144] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592586.927717] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:42 2020][8592586.932779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.939053] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:42 2020][8592586.945664] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592586.951931] ll_ost_io02_110 D ffff9c51be108000 0 40836 2 0x00000080 [Tue Mar 17 18:01:42 2020][8592586.959201] Call Trace: [Tue Mar 17 18:01:42 2020][8592586.961828] [] schedule+0x29/0x70 [Tue Mar 17 18:01:42 2020][8592586.966972] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.974037] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:42 2020][8592586.980050] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.987278] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:42 2020][8592586.993990] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.001731] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:42 2020][8592587.007922] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592587.014720] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.022037] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.029617] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.036758] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:42 2020][8592587.043031] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:42 2020][8592587.049828] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:42 2020][8592587.056043] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.062776] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.069423] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.076493] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:42 2020][8592587.083805] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:42 2020][8592587.089558] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:42 2020][8592587.096110] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.103552] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.110635] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.118381] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:42 2020][8592587.125634] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.133494] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.140458] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:42 2020][8592587.145887] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.152360] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.159935] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:42 2020][8592587.164995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592587.171282] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:42 2020][8592587.177900] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:42 2020][8592587.184174] ll_ost_io03_095 D ffff9c6b9c1b2080 0 40837 2 0x00000080 [Tue Mar 17 18:01:42 2020][8592587.191444] Call Trace: [Tue Mar 17 18:01:42 2020][8592587.194074] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:42 2020][8592587.200693] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:42 2020][8592587.207045] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:42 2020][8592587.212359] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:42 2020][8592587.217861] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.224828] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.232224] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:42 2020][8592587.238261] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:42 2020][8592587.244623] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:42 2020][8592587.251243] [] ? ldiskfs_mkdir+0x80/0x290 [ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.257947] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:42 2020][8592587.265085] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:42 2020][8592587.270312] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:42 2020][8592587.275977] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.283378] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:42 2020][8592587.290348] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.297662] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.304558] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:42 2020][8592587.311874] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:42 2020][8592587.318754] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:42 2020][8592587.324973] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.331703] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.338352] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.345402] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:42 2020][8592587.350929] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.358040] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.365804] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:42 2020][8592587.373060] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.380920] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.387887] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:42 2020][8592587.393323] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.399794] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:42 2020][8592587.407369] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:42 2020][8592587.412431] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.418706] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:43 2020][8592587.425324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.431606] ll_ost_io00_102 D ffff9c5a3a255140 0 40838 2 0x00000080 [Tue Mar 17 18:01:43 2020][8592587.438878] Call Trace: [Tue Mar 17 18:01:43 2020][8592587.441529] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.449186] [] schedule+0x29/0x70 [Tue Mar 17 18:01:43 2020][8592587.454336] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.461402] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:43 2020][8592587.467414] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.474642] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.481356] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.489097] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:43 2020][8592587.495299] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.500273] LustreError: dumping log to /tmp/lustre-log.1584493303.85263 [Tue Mar 17 18:01:43 2020][8592587.508964] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.516296] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.523872] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.531011] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:43 2020][8592587.537286] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:43 2020][8592587.544082] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:43 2020][8592587.550298] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.557030] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.563692] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.570763] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:43 2020][8592587.578077] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:43 2020][8592587.583833] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:43 2020][8592587.590393] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.597828] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.604915] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.612662] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:43 2020][8592587.619914] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.627775] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.634755] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:43 2020][8592587.640187] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.646662] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.654232] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:43 2020][8592587.659293] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.665558] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:43 2020][8592587.672172] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.678445] ll_ost_io03_096 D ffff9c5a3a256180 0 40839 2 0x00000080 [Tue Mar 17 18:01:43 2020][8592587.685715] Call Trace: [Tue Mar 17 18:01:43 2020][8592587.688343] [] schedule+0x29/0x70 [Tue Mar 17 18:01:43 2020][8592587.693486] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.700553] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:43 2020][8592587.706563] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.713790] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:43 2020][8592587.719633] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.726349] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.733663] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:43 2020][8592587.739852] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.746651] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.753968] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.761554] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.768698] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:43 2020][8592587.775491] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:43 2020][8592587.781709] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.788438] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.795086] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.802137] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:43 2020][8592587.807657] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.814734] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.822483] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:43 2020][8592587.829749] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.837606] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.844571] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:43 2020][8592587.850007] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.856482] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.864054] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:43 2020][8592587.869114] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.875388] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:43 2020][8592587.882001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592587.888268] ll_ost_io00_103 D ffff9c4f83764100 0 40840 2 0x00000080 [Tue Mar 17 18:01:43 2020][8592587.895554] Call Trace: [Tue Mar 17 18:01:43 2020][8592587.898181] [] schedule+0x29/0x70 [Tue Mar 17 18:01:43 2020][8592587.903326] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.910376] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:43 2020][8592587.916386] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.923613] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.930324] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.938066] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:43 2020][8592587.944249] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592587.951045] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.958362] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.965952] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592587.973095] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:43 2020][8592587.979368] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:43 2020][8592587.986163] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:43 2020][8592587.992380] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592587.999108] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.005759] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.012809] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:43 2020][8592588.018849] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.026291] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.033371] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.041119] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:43 2020][8592588.048372] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.056233] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.063198] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:43 2020][8592588.068635] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.075109] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.082680] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:43 2020][8592588.087742] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592588.094030] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:43 2020][8592588.100647] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592588.106920] ll_ost_io03_097 D ffff9c4f83762080 0 40841 2 0x00000080 [Tue Mar 17 18:01:43 2020][8592588.114190] Call Trace: [Tue Mar 17 18:01:43 2020][8592588.116826] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.124479] [] schedule+0x29/0x70 [Tue Mar 17 18:01:43 2020][8592588.129621] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.136672] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:43 2020][8592588.142684] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.149912] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.156622] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.164380] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:43 2020][8592588.170563] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.177362] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.184677] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.192249] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.199389] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:43 2020][8592588.205664] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:43 2020][8592588.212460] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:43 2020][8592588.218675] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.225421] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.232074] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.239125] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:43 2020][8592588.244650] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.251730] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.259477] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:43 2020][8592588.266731] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.274592] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.281554] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:43 2020][8592588.286983] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.293472] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:43 2020][8592588.301046] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:43 2020][8592588.306099] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592588.312365] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:43 2020][8592588.318979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:43 2020][8592588.325251] ll_ost_io02_111 D ffff9c4f83765140 0 40842 2 0x00000080 [Tue Mar 17 18:01:43 2020][8592588.332522] Call Trace: [Tue Mar 17 18:01:43 2020][8592588.335158] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.342811] [] schedule+0x29/0x70 [Tue Mar 17 18:01:43 2020][8592588.347954] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.355040] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:43 2020][8592588.361049] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.368278] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.374988] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.382731] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:43 2020][8592588.388922] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:43 2020][8592588.395718] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.403033] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.410608] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:43 2020][8592588.417749] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.424039] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:44 2020][8592588.430835] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.437053] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.443770] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:44 2020][8592588.449430] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:44 2020][8592588.455185] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:44 2020][8592588.460712] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.467792] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.475553] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:44 2020][8592588.482808] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.490668] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.497649] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:44 2020][8592588.503079] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.509553] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.517126] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:44 2020][8592588.522187] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.524271] LustreError: dumping log to /tmp/lustre-log.1584493304.40847 [Tue Mar 17 18:01:44 2020][8592588.535326] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:44 2020][8592588.541946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.548213] ll_ost_io00_104 D ffff9c4f83766180 0 40843 2 0x00000080 [Tue Mar 17 18:01:44 2020][8592588.555500] Call Trace: [Tue Mar 17 18:01:44 2020][8592588.558129] [] schedule+0x29/0x70 [Tue Mar 17 18:01:44 2020][8592588.563281] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.570330] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:44 2020][8592588.576339] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.583568] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.590277] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.598020] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:44 2020][8592588.604203] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.611001] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.618317] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.625907] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.633049] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.639321] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:44 2020][8592588.646115] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.652335] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.659074] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.665719] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.672772] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:44 2020][8592588.678812] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.686252] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.693333] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.701081] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:44 2020][8592588.708334] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.716196] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.723159] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:44 2020][8592588.728587] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.735062] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.742634] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:44 2020][8592588.747697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.753985] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:44 2020][8592588.760600] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.766867] ll_ost_io03_098 D ffff9c4f83761040 0 40844 2 0x00000080 [Tue Mar 17 18:01:44 2020][8592588.774143] Call Trace: [Tue Mar 17 18:01:44 2020][8592588.776773] [] schedule+0x29/0x70 [Tue Mar 17 18:01:44 2020][8592588.781924] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.788977] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:44 2020][8592588.794990] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.802219] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.808931] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.816690] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:44 2020][8592588.822873] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592588.829670] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.836987] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.844561] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592588.851700] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.857975] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:44 2020][8592588.864771] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:44 2020][8592588.870986] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.877717] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.884365] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.891430] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:44 2020][8592588.897465] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.904900] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.911979] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.919725] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:44 2020][8592588.926978] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.934837] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.941805] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:44 2020][8592588.947242] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.953715] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592588.961301] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:44 2020][8592588.966358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.972631] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:44 2020][8592588.979244] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592588.985511] ll_ost_io03_099 D ffff9c4f837630c0 0 40845 2 0x00000080 [Tue Mar 17 18:01:44 2020][8592588.992788] Call Trace: [Tue Mar 17 18:01:44 2020][8592588.995418] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:44 2020][8592589.002037] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:44 2020][8592589.008390] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:44 2020][8592589.013727] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:44 2020][8592589.019223] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.026198] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.033595] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:44 2020][8592589.039614] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:44 2020][8592589.045973] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:44 2020][8592589.052591] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:44 2020][8592589.058084] [] ? dqget+0x1a7/0x450 [Tue Mar 17 18:01:44 2020][8592589.063310] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:44 2020][8592589.070445] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:44 2020][8592589.075695] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:44 2020][8592589.081360] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.088762] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592589.095731] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.103045] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.109926] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.117243] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:44 2020][8592589.124120] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:44 2020][8592589.130338] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.137071] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.143731] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.150804] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:44 2020][8592589.158118] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:44 2020][8592589.163874] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:01:44 2020][8592589.169540] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:44 2020][8592589.175067] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.182145] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.189894] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:44 2020][8592589.197145] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.205008] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.211988] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:44 2020][8592589.217764] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.224237] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.231811] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:44 2020][8592589.236872] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592589.243146] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:44 2020][8592589.249760] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:44 2020][8592589.256032] ll_ost_io00_105 D ffff9c4a3e660000 0 40846 2 0x00000080 [Tue Mar 17 18:01:44 2020][8592589.263303] Call Trace: [Tue Mar 17 18:01:44 2020][8592589.265933] [] schedule+0x29/0x70 [Tue Mar 17 18:01:44 2020][8592589.271082] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:44 2020][8592589.278151] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:44 2020][8592589.284160] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592589.291385] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:44 2020][8592589.297229] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:44 2020][8592589.303945] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.311263] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:44 2020][8592589.317449] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:44 2020][8592589.324246] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.331563] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.339154] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:44 2020][8592589.346293] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:44 2020][8592589.353087] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:44 2020][8592589.359324] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.366034] [] ? __slab_free+0x81/0x2f0 [Tue Mar 17 18:01:44 2020][8592589.371699] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:44 2020][8592589.377447] [] ? __switch_to+0xce/0x580 [Tue Mar 17 18:01:44 2020][8592589.383115] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:44 2020][8592589.388639] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.395721] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.403481] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:44 2020][8592589.410740] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:44 2020][8592589.418598] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.425563] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:45 2020][8592589.431347] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.437823] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.445410] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:45 2020][8592589.450473] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.456747] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:45 2020][8592589.463382] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.469666] ll_ost_io03_100 D ffff9c4a3e666180 0 40847 2 0x00000080 [Tue Mar 17 18:01:45 2020][8592589.476937] Call Trace: [Tue Mar 17 18:01:45 2020][8592589.479575] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.487237] [] schedule+0x29/0x70 [Tue Mar 17 18:01:45 2020][8592589.492385] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:45 2020][8592589.499438] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:45 2020][8592589.505447] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:45 2020][8592589.512675] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:45 2020][8592589.519385] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.527128] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:45 2020][8592589.533326] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:45 2020][8592589.540126] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.547444] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.548282] LustreError: dumping log to /tmp/lustre-log.1584493305.68510 [Tue Mar 17 18:01:45 2020][8592589.561912] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.569055] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:45 2020][8592589.575328] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:45 2020][8592589.582124] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:45 2020][8592589.588339] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.595071] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.601719] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.608770] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:45 2020][8592589.614300] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.621384] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.629132] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:45 2020][8592589.636383] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.644246] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.651210] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:45 2020][8592589.656637] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.663113] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.670700] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:45 2020][8592589.675755] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.682029] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:45 2020][8592589.688643] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.694918] ll_ost_io00_106 D ffff9c4a3e661040 0 40848 2 0x00000080 [Tue Mar 17 18:01:45 2020][8592589.702195] Call Trace: [Tue Mar 17 18:01:45 2020][8592589.704822] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:45 2020][8592589.710829] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:45 2020][8592589.717442] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:45 2020][8592589.723802] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:45 2020][8592589.729130] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:45 2020][8592589.734627] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.741594] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.748990] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:45 2020][8592589.755272] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.763628] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:45 2020][8592589.769996] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.777224] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.784878] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:45 2020][8592589.790466] [] ? ldiskfs_ext_find_extent+0x134/0x340 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.798141] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.806066] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.813467] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.820523] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.828701] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:45 2020][8592589.835670] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.842897] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.850211] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:45 2020][8592589.857005] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:45 2020][8592589.863225] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.869976] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.876618] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.883670] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:45 2020][8592589.889709] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.897135] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.904214] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.911963] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:45 2020][8592589.919215] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.927075] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.934040] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:45 2020][8592589.939484] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.945961] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592589.953534] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:45 2020][8592589.958593] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.964868] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:45 2020][8592589.971480] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592589.977747] ll_ost_io03_101 D ffff9c4a3e662080 0 40849 2 0x00000080 [Tue Mar 17 18:01:45 2020][8592589.985024] Call Trace: [Tue Mar 17 18:01:45 2020][8592589.987652] [] schedule+0x29/0x70 [Tue Mar 17 18:01:45 2020][8592589.992801] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592589.999867] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.007005] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:45 2020][8592590.013023] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:45 2020][8592590.019212] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:45 2020][8592590.026007] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:45 2020][8592590.032224] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.038954] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.045603] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.052652] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:45 2020][8592590.058170] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.065267] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.073013] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:45 2020][8592590.080272] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.088136] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.095102] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:45 2020][8592590.100532] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.107004] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.114576] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:45 2020][8592590.119630] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592590.125896] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:45 2020][8592590.132516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592590.138797] ll_ost_io00_107 D ffff9c4a3e664100 0 40850 2 0x00000080 [Tue Mar 17 18:01:45 2020][8592590.146069] Call Trace: [Tue Mar 17 18:01:45 2020][8592590.148699] [] schedule+0x29/0x70 [Tue Mar 17 18:01:45 2020][8592590.153848] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:45 2020][8592590.160902] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:45 2020][8592590.166919] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:45 2020][8592590.174144] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:45 2020][8592590.179986] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:45 2020][8592590.186700] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.194008] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:45 2020][8592590.200214] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:45 2020][8592590.207013] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.214329] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.221904] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.229041] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:45 2020][8592590.235837] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:45 2020][8592590.242055] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.248784] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.255432] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.262483] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:45 2020][8592590.268015] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.275099] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.282846] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:45 2020][8592590.290098] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.297957] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.304926] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:45 2020][8592590.310362] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.316835] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.324406] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:45 2020][8592590.329473] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592590.335743] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:45 2020][8592590.342356] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:45 2020][8592590.348629] ll_ost_io00_108 D ffff9c4a3e6630c0 0 40851 2 0x00000080 [Tue Mar 17 18:01:45 2020][8592590.355898] Call Trace: [Tue Mar 17 18:01:45 2020][8592590.358529] [] schedule+0x29/0x70 [Tue Mar 17 18:01:45 2020][8592590.363683] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.370737] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:45 2020][8592590.377873] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:45 2020][8592590.383891] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:45 2020][8592590.390093] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:45 2020][8592590.396891] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:45 2020][8592590.403109] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.409839] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:45 2020][8592590.416488] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.423536] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:46 2020][8592590.429568] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.436995] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.444075] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.451837] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:46 2020][8592590.459107] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.466968] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.473934] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:46 2020][8592590.479362] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.485837] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.493409] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:46 2020][8592590.498468] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.504737] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:46 2020][8592590.511360] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.517632] ll_ost_io00_109 D ffff9c4a26975140 0 40852 2 0x00000080 [Tue Mar 17 18:01:46 2020][8592590.524917] Call Trace: [Tue Mar 17 18:01:46 2020][8592590.527545] [] schedule+0x29/0x70 [Tue Mar 17 18:01:46 2020][8592590.532690] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.539740] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:46 2020][8592590.545750] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.552975] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:46 2020][8592590.558820] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.565532] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.572274] LustreError: dumping log to /tmp/lustre-log.1584493306.68899 [Tue Mar 17 18:01:46 2020][8592590.579711] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:46 2020][8592590.585909] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.592709] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.600026] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.607598] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.614738] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:46 2020][8592590.621533] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:46 2020][8592590.627749] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.634480] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.641129] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.648180] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:46 2020][8592590.654235] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.661662] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.668741] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.676489] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:46 2020][8592590.683740] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.691601] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.698568] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:46 2020][8592590.704004] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.710479] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.718052] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:46 2020][8592590.723111] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.729391] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:46 2020][8592590.736008] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.742280] ll_ost_io03_102 D ffff9c4a26974100 0 40853 2 0x00000080 [Tue Mar 17 18:01:46 2020][8592590.749550] Call Trace: [Tue Mar 17 18:01:46 2020][8592590.752188] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.759851] [] schedule+0x29/0x70 [Tue Mar 17 18:01:46 2020][8592590.765002] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.772055] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:46 2020][8592590.778069] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.785311] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.792016] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.799761] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:46 2020][8592590.805951] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592590.812748] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.820063] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.827638] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.834777] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:46 2020][8592590.841051] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:46 2020][8592590.847846] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:46 2020][8592590.854078] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.860784] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:46 2020][8592590.866307] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.873387] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.881136] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:46 2020][8592590.888390] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.896250] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.903216] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:46 2020][8592590.908650] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.915124] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592590.922711] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:46 2020][8592590.927767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.934032] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:46 2020][8592590.940646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592590.946921] ll_ost_io03_103 D ffff9c4a269730c0 0 40854 2 0x00000080 [Tue Mar 17 18:01:46 2020][8592590.954189] Call Trace: [Tue Mar 17 18:01:46 2020][8592590.956818] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:46 2020][8592590.963431] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:46 2020][8592590.969783] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:46 2020][8592590.975096] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:46 2020][8592590.980604] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.987574] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592590.994970] [] __dquot_transfer+0x28d/0x510 [Tue Mar 17 18:01:46 2020][8592591.000990] [] ? lqe_locate+0x36/0x840 [lquota] [Tue Mar 17 18:01:46 2020][8592591.007350] [] ? qsd_op_begin0+0xc7/0x960 [lquota] [Tue Mar 17 18:01:46 2020][8592591.013969] [] ? __alloc_pages_direct_compact+0xfb/0x193 [Tue Mar 17 18:01:46 2020][8592591.021108] [] ? dqget+0x190/0x450 [Tue Mar 17 18:01:46 2020][8592591.026342] [] dquot_transfer+0x84/0xf0 [Tue Mar 17 18:01:46 2020][8592591.032007] [] osd_quota_transfer+0x16d/0x230 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.039407] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592591.046378] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.053708] [] osd_attr_set+0x11f/0xb90 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.060591] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.067904] [] ofd_commitrw_write+0x13c2/0x1d40 [ofd] [Tue Mar 17 18:01:46 2020][8592591.074786] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:46 2020][8592591.081003] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.087732] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.094380] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.101452] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:46 2020][8592591.108763] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:46 2020][8592591.114533] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:46 2020][8592591.120054] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.127133] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.134881] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:46 2020][8592591.142134] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.149993] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.156959] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:46 2020][8592591.162396] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.168869] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.176443] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:46 2020][8592591.181515] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592591.187788] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:46 2020][8592591.194408] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592591.200673] ll_ost_io00_110 D ffff9c4a26972080 0 40855 2 0x00000080 [Tue Mar 17 18:01:46 2020][8592591.207951] Call Trace: [Tue Mar 17 18:01:46 2020][8592591.210581] [] schedule+0x29/0x70 [Tue Mar 17 18:01:46 2020][8592591.215731] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:46 2020][8592591.222781] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:46 2020][8592591.228791] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592591.236017] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:46 2020][8592591.241862] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:46 2020][8592591.248588] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.255900] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:46 2020][8592591.262089] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:46 2020][8592591.268886] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.276204] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.283775] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:46 2020][8592591.290917] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:46 2020][8592591.297710] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:46 2020][8592591.303928] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.310672] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.317322] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.324393] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:46 2020][8592591.331708] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:46 2020][8592591.337462] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:46 2020][8592591.342988] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.350070] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.357817] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:46 2020][8592591.365085] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.372946] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.379925] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:46 2020][8592591.385356] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.391830] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:46 2020][8592591.399401] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:46 2020][8592591.404454] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:46 2020][8592591.410722] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:46 2020][8592591.417343] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.423617] ll_ost_io00_111 D ffff9c4a26971040 0 40856 2 0x00000080 [Tue Mar 17 18:01:47 2020][8592591.430893] Call Trace: [Tue Mar 17 18:01:47 2020][8592591.433523] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:47 2020][8592591.438845] [] schedule+0x29/0x70 [Tue Mar 17 18:01:47 2020][8592591.444005] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:47 2020][8592591.450033] [] ? __find_get_block+0xbc/0x120 [Tue Mar 17 18:01:47 2020][8592591.456127] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:47 2020][8592591.461551] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:47 2020][8592591.467396] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:47 2020][8592591.472715] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:47 2020][8592591.478929] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:47 2020][8592591.484337] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:47 2020][8592591.489743] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:47 2020][8592591.495776] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:47 2020][8592591.501089] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:47 2020][8592591.507969] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.514077] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:47 2020][8592591.519660] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:47 2020][8592591.526538] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:47 2020][8592591.534130] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.542395] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:47 2020][8592591.547907] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.554899] [] osd_punch+0x28c/0x6b0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.561520] [] ofd_object_punch+0x7df/0xd30 [ofd] [Tue Mar 17 18:01:47 2020][8592591.568066] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:47 2020][8592591.574373] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.581467] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.589218] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:47 2020][8592591.596483] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.604340] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.611317] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:47 2020][8592591.616748] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.623225] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.630822] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:47 2020][8592591.635884] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.642171] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:47 2020][8592591.648787] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.655053] ll_ost_io03_104 D ffff9c4a26976180 0 40857 2 0x00000080 [Tue Mar 17 18:01:47 2020][8592591.662339] Call Trace: [Tue Mar 17 18:01:47 2020][8592591.664969] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:47 2020][8592591.671589] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:47 2020][8592591.677956] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:47 2020][8592591.683291] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:47 2020][8592591.688792] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.695773] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.703172] [] __dquot_alloc_space+0x1f8/0x260 [Tue Mar 17 18:01:47 2020][8592591.709453] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.716349] [] ? ldiskfs_block_bitmap_csum_verify+0xe0/0x140 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.724707] [] ldiskfs_mb_new_blocks+0xe1/0xb20 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.731934] [] ? ldiskfs_es_insert_extent+0xdb/0x1c0 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.739608] [] ? __es_tree_search+0x60/0x80 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.746488] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:01:47 2020][8592591.752075] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.759766] [] ? ldiskfs_ext_find_extent+0x292/0x340 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.767433] [] ? ldiskfs_ext_put_gap_in_cache+0x83/0x90 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.775355] [] ldiskfs_ext_map_blocks+0x4a6/0xf60 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.782769] [] ldiskfs_map_blocks+0x155/0x700 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.789829] [] osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.798019] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:47 2020][8592591.804986] [] osd_write_commit+0x3a2/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.812229] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592591.819542] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:47 2020][8592591.826339] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:47 2020][8592591.832585] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.839320] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.845968] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.853053] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:47 2020][8592591.860359] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:47 2020][8592591.866104] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:47 2020][8592591.872669] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.880100] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.887179] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.894956] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:47 2020][8592591.902207] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.910067] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.917046] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:47 2020][8592591.922478] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.928952] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592591.936536] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:47 2020][8592591.941595] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.947881] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:47 2020][8592591.954495] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592591.960791] ll_ost_io03_105 D ffff9c5a2ab8e180 0 40858 2 0x00000080 [Tue Mar 17 18:01:47 2020][8592591.968066] Call Trace: [Tue Mar 17 18:01:47 2020][8592591.970697] [] schedule+0x29/0x70 [Tue Mar 17 18:01:47 2020][8592591.975846] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:47 2020][8592591.982912] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:47 2020][8592591.988924] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:47 2020][8592591.996167] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.002870] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.010630] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:47 2020][8592592.016814] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.023611] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.030943] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.038519] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.045660] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:47 2020][8592592.051947] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:01:47 2020][8592592.058483] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:47 2020][8592592.064803] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.071886] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.079634] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:47 2020][8592592.086899] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.094777] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.101746] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:47 2020][8592592.107196] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.108285] LustreError: dumping log to /tmp/lustre-log.1584493307.68865 [Tue Mar 17 18:01:47 2020][8592592.120554] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.128140] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:47 2020][8592592.133198] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592592.139487] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:47 2020][8592592.146101] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592592.152377] ll_ost_io03_106 D ffff9c5a2ab8d140 0 40859 2 0x00000080 [Tue Mar 17 18:01:47 2020][8592592.159689] Call Trace: [Tue Mar 17 18:01:47 2020][8592592.162324] [] ? __read_extent_tree_block+0x55/0x1e0 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.169980] [] schedule+0x29/0x70 [Tue Mar 17 18:01:47 2020][8592592.175145] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.182199] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:47 2020][8592592.188216] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.195459] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.202174] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.209929] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:47 2020][8592592.216116] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:47 2020][8592592.222940] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.230252] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.237842] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.244986] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:47 2020][8592592.251260] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:47 2020][8592592.258067] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:47 2020][8592592.264282] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.271017] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.277681] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.284734] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:47 2020][8592592.290793] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.298227] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.305307] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.313068] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:47 2020][8592592.320322] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.328182] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.335166] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:47 2020][8592592.340603] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.347079] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:47 2020][8592592.354664] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:47 2020][8592592.359718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592592.366013] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:47 2020][8592592.372634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:47 2020][8592592.378906] ll_ost_io03_107 D ffff9c598da84100 0 40860 2 0x00000080 [Tue Mar 17 18:01:47 2020][8592592.386191] Call Trace: [Tue Mar 17 18:01:47 2020][8592592.388821] [] schedule+0x29/0x70 [Tue Mar 17 18:01:47 2020][8592592.393967] [] osd_trans_stop+0x265/0x8e0 [osd_ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.401036] [] ? ldiskfs_dirty_inode+0x54/0x60 [ldiskfs] [Tue Mar 17 18:01:47 2020][8592592.408174] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:47 2020][8592592.414199] [] ofd_trans_stop+0x25/0x60 [ofd] [Tue Mar 17 18:01:47 2020][8592592.420406] [] ofd_commitrw_write+0x9d4/0x1d40 [ofd] [Tue Mar 17 18:01:48 2020][8592592.427201] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:48 2020][8592592.433435] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.440167] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.446817] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.453912] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:48 2020][8592592.461224] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:48 2020][8592592.466995] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:48 2020][8592592.473548] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.480983] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.488105] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.495853] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:48 2020][8592592.503120] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.510983] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.517946] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:48 2020][8592592.523392] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.529867] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.537439] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:48 2020][8592592.542514] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592592.548784] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:48 2020][8592592.555404] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592592.561699] ll_ost_io03_108 D ffff9c598da80000 0 40861 2 0x00000080 [Tue Mar 17 18:01:48 2020][8592592.568974] Call Trace: [Tue Mar 17 18:01:48 2020][8592592.571604] [] schedule+0x29/0x70 [Tue Mar 17 18:01:48 2020][8592592.576754] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.583819] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:48 2020][8592592.589832] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.597076] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.603788] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.604282] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.238@o2ib7: 10 seconds [Tue Mar 17 18:01:48 2020][8592592.604286] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 347 previous similar messages [Tue Mar 17 18:01:48 2020][8592592.620294] LustreError: dumping log to /tmp/lustre-log.1584493308.98683 [Tue Mar 17 18:01:48 2020][8592592.638442] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:48 2020][8592592.644635] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.651429] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.658760] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.666337] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.673475] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:48 2020][8592592.679764] [] ofd_commitrw_write+0xa31/0x1d40 [ofd] [Tue Mar 17 18:01:48 2020][8592592.686562] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:48 2020][8592592.692780] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.699539] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.706183] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.713253] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:48 2020][8592592.720580] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:48 2020][8592592.726330] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:48 2020][8592592.732895] [] ? target_send_reply_msg+0x170/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.740326] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.747419] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.755168] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:48 2020][8592592.762451] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.770307] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.777292] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:48 2020][8592592.782728] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.789203] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.796788] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:48 2020][8592592.801845] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592592.808134] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:48 2020][8592592.814747] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592592.821014] ll_ost_io03_109 D ffff9c598da81040 0 40862 2 0x00000080 [Tue Mar 17 18:01:48 2020][8592592.828318] Call Trace: [Tue Mar 17 18:01:48 2020][8592592.830946] [] schedule+0x29/0x70 [Tue Mar 17 18:01:48 2020][8592592.836097] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.843164] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:48 2020][8592592.849177] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.856418] [] ? crypto_mod_get+0x19/0x40 [Tue Mar 17 18:01:48 2020][8592592.862263] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.868990] [] ? osd_declare_qid+0x200/0x4a0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.876299] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:48 2020][8592592.882498] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592592.889309] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.896629] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.904217] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592592.911359] [] ofd_commitrw_write+0xf1e/0x1d40 [ofd] [Tue Mar 17 18:01:48 2020][8592592.918154] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:48 2020][8592592.924388] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.931092] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:48 2020][8592592.936617] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.943711] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.951460] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:48 2020][8592592.958713] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.966602] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.973567] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:48 2020][8592592.979002] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.985494] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592592.993067] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:48 2020][8592592.998128] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592593.004418] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:48 2020][8592593.011033] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592593.017307] ll_ost_io03_110 D ffff9c7312860000 0 40863 2 0x00000080 [Tue Mar 17 18:01:48 2020][8592593.024602] Call Trace: [Tue Mar 17 18:01:48 2020][8592593.027234] [] ? __percpu_counter_add+0x51/0x70 [Tue Mar 17 18:01:48 2020][8592593.033591] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:48 2020][8592593.039092] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:48 2020][8592593.044408] [] schedule+0x29/0x70 [Tue Mar 17 18:01:48 2020][8592593.049556] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:48 2020][8592593.055585] [] ? dquot_get_dqblk+0x144/0x1f0 [Tue Mar 17 18:01:48 2020][8592593.061689] [] ? osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.069514] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:48 2020][8592593.074843] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:48 2020][8592593.081038] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:48 2020][8592593.086441] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:48 2020][8592593.091853] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:48 2020][8592593.097864] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:48 2020][8592593.103184] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:48 2020][8592593.110072] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592593.116169] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:48 2020][8592593.121753] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.128646] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:48 2020][8592593.134051] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.141657] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.149923] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.157601] [] ? osd_ldiskfs_map_inode_pages+0x143/0x420 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.165957] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.173097] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.180599] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.187572] [] osd_write_commit+0x41f/0x8c0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.194797] [] ? osd_trans_start+0x235/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.202126] [] ofd_commitrw_write+0xfbe/0x1d40 [ofd] [Tue Mar 17 18:01:48 2020][8592593.208923] [] ofd_commitrw+0x48c/0x9e0 [ofd] [Tue Mar 17 18:01:48 2020][8592593.215140] [] tgt_brw_write+0x10cb/0x1cf0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.221903] [] ? lustre_msg_buf+0x17/0x60 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.228555] [] ? __req_capsule_get+0x163/0x740 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.235622] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:48 2020][8592593.242950] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:48 2020][8592593.248701] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:01:48 2020][8592593.254228] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.261324] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.269071] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:48 2020][8592593.276323] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.284212] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.291176] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:48 2020][8592593.296613] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.303101] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:48 2020][8592593.310669] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:48 2020][8592593.315742] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592593.322013] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:48 2020][8592593.328634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:48 2020][8592593.334922] ll_ost_io03_111 D ffff9c7312861040 0 40864 2 0x00000080 [Tue Mar 17 18:01:48 2020][8592593.342195] Call Trace: [Tue Mar 17 18:01:48 2020][8592593.344823] [] schedule+0x29/0x70 [Tue Mar 17 18:01:48 2020][8592593.349973] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.357074] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:48 2020][8592593.363087] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.370330] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.377043] [] ? osd_declare_xattr_set+0xf1/0x3a0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.384787] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:48 2020][8592593.390991] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:48 2020][8592593.397790] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.405108] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.412717] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:48 2020][8592593.419855] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:48 2020][8592593.426127] [] ofd_object_punch+0x73d/0xd30 [ofd] [Tue Mar 17 18:01:49 2020][8592593.432678] [] ofd_punch_hdl+0x493/0xa30 [ofd] [Tue Mar 17 18:01:49 2020][8592593.438986] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.446065] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.453839] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:49 2020][8592593.461091] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.468980] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.475941] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:49 2020][8592593.481379] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.487881] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.495450] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592593.500503] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.506786] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592593.513408] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.519681] ldlm_cn00_031 D ffff9c7a2da5a080 0 60459 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592593.526968] Call Trace: [Tue Mar 17 18:01:49 2020][8592593.529599] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592593.534745] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:49 2020][8592593.541222] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.547320] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:49 2020][8592593.553084] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:49 2020][8592593.558588] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.565042] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.571506] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.578907] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.585963] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.592947] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.600100] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.607421] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592593.614518] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.622020] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.629087] [] ? LNetPut+0x2cc/0xb50 [lnet] [Tue Mar 17 18:01:49 2020][8592593.635130] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.642455] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.644286] LustreError: dumping log to /tmp/lustre-log.1584493309.73705 [Tue Mar 17 18:01:49 2020][8592593.655977] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:49 2020][8592593.662355] [] ofd_lvbo_update+0x577/0xdf0 [ofd] [Tue Mar 17 18:01:49 2020][8592593.668833] [] ? lustre_pack_reply_v2+0x135/0x290 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.676188] [] ? class_handle2object+0xb9/0x1c0 [obdclass] [Tue Mar 17 18:01:49 2020][8592593.683527] [] ldlm_request_cancel+0x404/0x780 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.690610] [] ldlm_handle_cancel+0x232/0x2b0 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.697618] [] ldlm_cancel_handler+0x158/0x590 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.704693] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.712553] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.719535] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:49 2020][8592593.724965] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.731439] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.739026] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592593.744083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.750384] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592593.757004] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.763281] ll_ost01_091 S ffff9c5a39eaa080 0 101301 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592593.770660] Call Trace: [Tue Mar 17 18:01:49 2020][8592593.773290] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:49 2020][8592593.779389] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592593.784571] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.791447] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:49 2020][8592593.797228] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.803731] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.811300] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592593.816368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.822637] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592593.829257] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.835539] ll_ost01_092 S ffff9c69c06e2080 0 101302 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592593.842897] Call Trace: [Tue Mar 17 18:01:49 2020][8592593.845526] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:49 2020][8592593.851626] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592593.856809] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.863698] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:49 2020][8592593.869474] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.875961] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.883529] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592593.888590] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.894873] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592593.901487] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.907761] ll_ost01_093 S ffff9c69c06e1040 0 101303 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592593.915133] Call Trace: [Tue Mar 17 18:01:49 2020][8592593.917762] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:49 2020][8592593.923855] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592593.929051] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.935928] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:49 2020][8592593.941703] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.948193] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592593.955766] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592593.960819] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.967107] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592593.973724] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592593.980000] ll_ost01_094 S ffff9c5ae945b0c0 0 101304 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592593.987387] Call Trace: [Tue Mar 17 18:01:49 2020][8592593.990016] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:49 2020][8592593.996109] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592594.001295] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.008172] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:49 2020][8592594.013964] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.020440] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.028026] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592594.033083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.039359] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592594.046005] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.052279] ll_ost01_095 D ffff9c68eef20000 0 101305 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592594.059643] Call Trace: [Tue Mar 17 18:01:49 2020][8592594.062291] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:49 2020][8592594.068303] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592594.074931] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:49 2020][8592594.081290] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:49 2020][8592594.086642] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.093649] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.100730] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.108477] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:49 2020][8592594.115757] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.123617] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.130599] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:49 2020][8592594.136027] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.142502] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.150089] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592594.155144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.161427] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592594.168041] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.174314] ll_ost01_096 D ffff9c68eef24100 0 101306 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592594.181695] Call Trace: [Tue Mar 17 18:01:49 2020][8592594.184325] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592594.189469] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:49 2020][8592594.196532] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:49 2020][8592594.202545] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:49 2020][8592594.209789] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:49 2020][8592594.216503] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592594.223998] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:49 2020][8592594.230184] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:49 2020][8592594.236981] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592594.244324] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:49 2020][8592594.251899] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:49 2020][8592594.259060] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:49 2020][8592594.265337] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:49 2020][8592594.271437] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:49 2020][8592594.278183] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.285952] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.293965] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:49 2020][8592594.300454] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.307536] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.315313] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:49 2020][8592594.322563] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.330423] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.337402] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:49 2020][8592594.342834] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.349324] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.356917] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:49 2020][8592594.361976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.368266] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:49 2020][8592594.374896] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:49 2020][8592594.381165] ll_ost01_097 S ffff9c68eef25140 0 101307 2 0x00000080 [Tue Mar 17 18:01:49 2020][8592594.388545] Call Trace: [Tue Mar 17 18:01:49 2020][8592594.391173] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:49 2020][8592594.397266] [] schedule+0x29/0x70 [Tue Mar 17 18:01:49 2020][8592594.402450] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.409331] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:49 2020][8592594.415116] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:49 2020][8592594.421604] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.429177] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.434259] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.440530] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.447145] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.453433] ll_ost01_098 S ffff9c68eef22080 0 101308 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.460809] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.463436] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592594.469530] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.474714] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.481594] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592594.487379] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.493881] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.501449] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.506516] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.512787] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.519405] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.525687] ll_ost01_099 S ffff9c68eef21040 0 101309 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.533045] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.535674] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592594.541775] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.546959] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.553855] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592594.559631] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.566120] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.573687] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.578748] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.585028] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.591645] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.597919] ll_ost01_100 S ffff9c6a01feb0c0 0 101310 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.605299] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.607928] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592594.614021] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.619219] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.626095] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592594.631878] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.638366] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.645941] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.651001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.657289] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.663906] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.670173] ll_ost01_101 D ffff9c6a2f215140 0 101311 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.677554] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.680184] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:50 2020][8592594.686202] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.692835] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:50 2020][8592594.699190] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:50 2020][8592594.704549] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.711545] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.718638] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.726389] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:50 2020][8592594.733657] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.741537] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.748504] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:50 2020][8592594.753956] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.760432] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.768003] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.773079] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.779348] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.785961] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.792250] ll_ost01_102 S ffff9c6a341f8000 0 101312 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.799607] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.802235] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592594.808328] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.813528] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.820402] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592594.826179] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.832665] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.840231] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592594.845298] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.851568] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592594.858188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592594.864483] ll_ost01_103 D ffff9c6446fca080 0 101313 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592594.871845] Call Trace: [Tue Mar 17 18:01:50 2020][8592594.874474] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592594.879617] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:50 2020][8592594.886681] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:50 2020][8592594.892695] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:50 2020][8592594.899937] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:50 2020][8592594.906645] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:50 2020][8592594.914149] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:50 2020][8592594.920333] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:50 2020][8592594.927131] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:50 2020][8592594.934459] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:50 2020][8592594.942038] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:50 2020][8592594.949177] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:50 2020][8592594.955467] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:50 2020][8592594.961569] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:50 2020][8592594.968305] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.976091] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.984096] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:50 2020][8592594.990572] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592594.997679] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.005423] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:50 2020][8592595.012690] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.020547] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.027512] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:50 2020][8592595.032964] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.039438] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.047012] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.052085] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.058357] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592595.065005] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.071283] ll_ost01_104 S ffff9c6446fcc100 0 101314 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592595.078642] Call Trace: [Tue Mar 17 18:01:50 2020][8592595.081274] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592595.087389] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592595.092573] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.099454] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592595.105252] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.111729] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.119302] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.124390] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.130664] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592595.137286] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.143574] ll_ost01_105 S ffff9c6446fc9040 0 101315 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592595.150932] Call Trace: [Tue Mar 17 18:01:50 2020][8592595.153574] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592595.159669] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592595.164852] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.171726] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592595.177509] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.180295] LustreError: dumping log to /tmp/lustre-log.1584493310.68379 [Tue Mar 17 18:01:50 2020][8592595.190890] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.198477] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.203532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.209823] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592595.216436] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.222718] ll_ost00_079 S ffff9c6446fc8000 0 101316 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592595.230085] Call Trace: [Tue Mar 17 18:01:50 2020][8592595.232714] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592595.238815] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592595.244021] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.250898] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592595.256680] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.263168] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.270735] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.275797] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.282086] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592595.288701] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.294977] ll_ost01_106 S ffff9c6446fcd140 0 101317 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592595.302356] Call Trace: [Tue Mar 17 18:01:50 2020][8592595.304984] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592595.311079] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592595.316285] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.323168] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592595.328958] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.335434] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.343021] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.348076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.354356] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:50 2020][8592595.360971] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.367237] ll_ost00_080 S ffff9c6a0225a080 0 101318 2 0x00000080 [Tue Mar 17 18:01:50 2020][8592595.374610] Call Trace: [Tue Mar 17 18:01:50 2020][8592595.377240] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:50 2020][8592595.383342] [] schedule+0x29/0x70 [Tue Mar 17 18:01:50 2020][8592595.388530] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.395406] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:50 2020][8592595.401190] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.407676] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:50 2020][8592595.415242] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:50 2020][8592595.420297] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:50 2020][8592595.426600] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.433219] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.439493] ll_ost01_107 S ffff9c6a02258000 0 101319 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.446865] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.449496] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:51 2020][8592595.455595] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.460792] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.467671] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592595.473453] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.479942] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.487542] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.492602] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.498883] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.505499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.511772] ll_ost01_108 S ffff9c6a0225e180 0 101320 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.519146] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.521775] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:51 2020][8592595.527877] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.533068] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.539950] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592595.545733] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.552234] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.559805] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.564865] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.571148] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.577762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.584038] ll_ost00_081 S ffff9c6a0225c100 0 101321 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.591417] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.594046] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:51 2020][8592595.600150] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.605354] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.612228] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592595.618020] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.624496] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.632067] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.637134] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.643403] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.650016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.656304] ll_ost01_109 D ffff9c66a0952080 0 101322 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.663663] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.666291] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.671436] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:51 2020][8592595.678513] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:51 2020][8592595.684532] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:51 2020][8592595.691774] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:51 2020][8592595.692291] LustreError: dumping log to /tmp/lustre-log.1584493311.68372 [Tue Mar 17 18:01:51 2020][8592595.705386] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:51 2020][8592595.712870] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:51 2020][8592595.719062] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:51 2020][8592595.725858] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:51 2020][8592595.733187] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592595.740765] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:51 2020][8592595.747903] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:51 2020][8592595.754177] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:51 2020][8592595.760280] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:51 2020][8592595.767020] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.774817] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.782831] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:51 2020][8592595.789328] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.796417] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.804170] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:51 2020][8592595.811430] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.819293] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.826285] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:51 2020][8592595.831719] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.838205] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.845780] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.850856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.857128] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.863749] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.870050] ll_ost00_082 S ffff9c66a0955140 0 101323 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.877419] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.880065] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:51 2020][8592595.886161] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.891353] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.898232] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592595.904015] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.910489] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.918062] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.923127] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.929410] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592595.936028] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592595.942300] ll_ost02_099 S ffff9c66a0954100 0 101324 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592595.949673] Call Trace: [Tue Mar 17 18:01:51 2020][8592595.952304] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:51 2020][8592595.958404] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592595.963576] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.970452] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592595.976233] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.982707] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592595.990295] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592595.995352] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.001624] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592596.008237] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.014504] ll_ost01_110 D ffff9c66a0950000 0 101325 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592596.021884] Call Trace: [Tue Mar 17 18:01:51 2020][8592596.024514] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:51 2020][8592596.029825] [] schedule+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592596.034966] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:51 2020][8592596.040987] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:51 2020][8592596.048227] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:01:51 2020][8592596.053981] [] ? account_entity_dequeue+0xae/0xd0 [Tue Mar 17 18:01:51 2020][8592596.060534] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:01:51 2020][8592596.066374] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:51 2020][8592596.071686] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:51 2020][8592596.077866] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:51 2020][8592596.083267] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:51 2020][8592596.088672] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:51 2020][8592596.094244] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:51 2020][8592596.099560] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:51 2020][8592596.106020] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.112131] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:51 2020][8592596.119010] [] ? __getblk+0x2d/0x300 [Tue Mar 17 18:01:51 2020][8592596.124419] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:51 2020][8592596.132002] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.140272] [] ldiskfs_reserve_inode_write+0x70/0xa0 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.147931] [] ? jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:51 2020][8592596.154903] [] ? ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.162058] [] ldiskfs_mark_inode_dirty+0x53/0x210 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.169549] [] ldiskfs_dirty_inode+0x40/0x60 [ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.176518] [] osd_ref_del+0x19d/0x6a0 [osd_ldiskfs] [Tue Mar 17 18:01:51 2020][8592596.183325] [] ofd_destroy+0x509/0x960 [ofd] [Tue Mar 17 18:01:51 2020][8592596.189427] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:51 2020][8592596.196160] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.203947] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.211954] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:51 2020][8592596.218432] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.225509] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.233257] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:51 2020][8592596.240509] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.248385] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.255353] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:51 2020][8592596.261149] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.267619] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.275189] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592596.280243] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.286508] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592596.293122] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.299412] ll_ost01_111 D ffff9c66a0956180 0 101326 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592596.306769] Call Trace: [Tue Mar 17 18:01:51 2020][8592596.309398] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:51 2020][8592596.315406] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:51 2020][8592596.322040] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:51 2020][8592596.328396] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:51 2020][8592596.333752] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.340766] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.347849] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.355611] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:51 2020][8592596.362864] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.370727] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.377720] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:51 2020][8592596.383155] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.389631] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:51 2020][8592596.397201] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:51 2020][8592596.402253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.408534] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:51 2020][8592596.415149] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:51 2020][8592596.421417] ll_ost02_100 D ffff9c6a2e61e180 0 101327 2 0x00000080 [Tue Mar 17 18:01:51 2020][8592596.428779] Call Trace: [Tue Mar 17 18:01:51 2020][8592596.431409] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:52 2020][8592596.437424] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592596.444060] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:52 2020][8592596.450416] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:52 2020][8592596.455770] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.462778] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.469859] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.477606] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:52 2020][8592596.484872] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.492729] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.499695] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:52 2020][8592596.505159] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.511631] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.519202] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592596.524267] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.530537] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592596.537151] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.543430] ll_ost00_083 S ffff9c6a2e61d140 0 101328 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592596.550789] Call Trace: [Tue Mar 17 18:01:52 2020][8592596.553419] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:52 2020][8592596.559521] [] schedule+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592596.564712] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.571594] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:52 2020][8592596.577377] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.583864] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.591432] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592596.596493] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.602781] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592596.609398] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.615684] ll_ost02_101 D ffff9c6a2e619040 0 101329 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592596.623043] Call Trace: [Tue Mar 17 18:01:52 2020][8592596.625674] [] schedule+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592596.630827] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:52 2020][8592596.637902] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:52 2020][8592596.643919] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:52 2020][8592596.651162] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:52 2020][8592596.657874] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.665361] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:52 2020][8592596.671549] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:52 2020][8592596.678345] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.685664] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.693264] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.700403] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:52 2020][8592596.706676] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:52 2020][8592596.712777] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:52 2020][8592596.716299] LustreError: dumping log to /tmp/lustre-log.1584493312.68373 [Tue Mar 17 18:01:52 2020][8592596.726398] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.734178] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.742184] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:52 2020][8592596.748662] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.755740] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.763503] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:52 2020][8592596.770759] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.778634] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.785601] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:52 2020][8592596.791383] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.797873] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.805447] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592596.810499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.816780] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592596.823396] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592596.829670] ll_ost02_102 D ffff9c6a2e61a080 0 101330 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592596.837051] Call Trace: [Tue Mar 17 18:01:52 2020][8592596.839681] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592596.846317] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:52 2020][8592596.852671] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:52 2020][8592596.857988] [] dquot_commit+0x29/0xd0 [Tue Mar 17 18:01:52 2020][8592596.863501] [] ldiskfs_write_dquot+0x6c/0x90 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.870472] [] ldiskfs_mark_dquot_dirty+0x3f/0x60 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.877868] [] dquot_free_inode+0xf0/0x190 [Tue Mar 17 18:01:52 2020][8592596.883810] [] ? __dquot_initialize+0x3a/0x240 [Tue Mar 17 18:01:52 2020][8592596.890093] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:52 2020][8592596.895415] [] ldiskfs_free_inode+0xf7/0x610 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.902396] [] ? ldiskfs_mark_iloc_dirty+0x68/0x80 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.909887] [] ? ldiskfs_evict_inode+0x472/0x630 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.917200] [] ? ldiskfs_mark_inode_dirty+0x6f/0x210 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.924876] [] ldiskfs_evict_inode+0x57d/0x630 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.932018] [] evict+0xb4/0x180 [Tue Mar 17 18:01:52 2020][8592596.936991] [] iput+0xfc/0x190 [Tue Mar 17 18:01:52 2020][8592596.941898] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592596.949241] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:01:52 2020][8592596.956645] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:01:52 2020][8592596.963828] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:01:52 2020][8592596.970447] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:01:52 2020][8592596.977179] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.984968] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592596.992974] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:52 2020][8592596.999465] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.006547] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.014294] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:52 2020][8592597.021560] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.029432] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.036414] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:52 2020][8592597.041845] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.048318] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.055903] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592597.060959] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.067249] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592597.073864] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.080129] ll_ost02_103 D ffff9c7b0f994100 0 101331 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592597.087502] Call Trace: [Tue Mar 17 18:01:52 2020][8592597.090132] [] schedule+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592597.095291] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:52 2020][8592597.102357] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:52 2020][8592597.108370] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:52 2020][8592597.115613] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:52 2020][8592597.122329] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592597.129832] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:52 2020][8592597.136018] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:52 2020][8592597.142827] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592597.150146] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:52 2020][8592597.157748] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:52 2020][8592597.164887] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:52 2020][8592597.171178] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:52 2020][8592597.177281] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:52 2020][8592597.184009] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.191789] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.199797] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:52 2020][8592597.206289] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.213368] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.221131] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:52 2020][8592597.228387] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.236275] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.243238] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:52 2020][8592597.248667] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.255155] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.262731] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592597.267807] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.274075] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592597.280696] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.286985] ll_ost02_104 S ffff9c7b0f992080 0 101332 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592597.294344] Call Trace: [Tue Mar 17 18:01:52 2020][8592597.297001] [] schedule+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592597.302175] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.309067] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:52 2020][8592597.314846] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.321334] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.328908] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:52 2020][8592597.333976] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.340244] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:52 2020][8592597.346857] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:52 2020][8592597.353152] ll_ost00_084 S ffff9c7b0f9930c0 0 101333 2 0x00000080 [Tue Mar 17 18:01:52 2020][8592597.360514] Call Trace: [Tue Mar 17 18:01:52 2020][8592597.363144] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:52 2020][8592597.369265] [] schedule+0x29/0x70 [Tue Mar 17 18:01:52 2020][8592597.374453] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.381337] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:52 2020][8592597.387971] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:52 2020][8592597.393749] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.400224] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:52 2020][8592597.407810] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592597.412863] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.419158] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592597.425776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.432045] ll_ost00_085 S ffff9c7b0f996180 0 101334 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592597.439425] Call Trace: [Tue Mar 17 18:01:53 2020][8592597.442053] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592597.448153] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592597.453338] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.460233] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592597.466011] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.472515] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.480083] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592597.485144] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.491436] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592597.498047] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.504314] ll_ost00_086 S ffff9c7b0f990000 0 101335 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592597.511687] Call Trace: [Tue Mar 17 18:01:53 2020][8592597.514316] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592597.520409] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592597.525600] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.532488] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592597.538266] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.544754] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.552319] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592597.557387] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.563657] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592597.570276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.576558] ll_ost02_105 D ffff9c7b0f995140 0 101336 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592597.583916] Call Trace: [Tue Mar 17 18:01:53 2020][8592597.586544] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592597.591695] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:53 2020][8592597.598777] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:53 2020][8592597.604790] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:53 2020][8592597.612034] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:53 2020][8592597.618751] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:53 2020][8592597.626239] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:53 2020][8592597.632445] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:53 2020][8592597.639244] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:53 2020][8592597.646561] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:53 2020][8592597.654149] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:53 2020][8592597.661309] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:53 2020][8592597.667582] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:53 2020][8592597.673699] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:53 2020][8592597.680431] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.688196] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.696216] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:53 2020][8592597.702694] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.709773] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.717537] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:53 2020][8592597.724790] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.732651] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.739646] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:53 2020][8592597.745081] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.751568] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.759143] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592597.764218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.770488] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592597.777107] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.783388] ll_ost00_087 D ffff9c8a393b8000 0 101337 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592597.790748] Call Trace: [Tue Mar 17 18:01:53 2020][8592597.793377] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:53 2020][8592597.799392] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592597.806039] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:53 2020][8592597.812401] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:53 2020][8592597.817754] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.824766] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.831844] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.839591] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:53 2020][8592597.846859] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.854722] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.861717] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:53 2020][8592597.867152] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.873637] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.881206] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592597.886265] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.892547] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592597.899162] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592597.905430] ll_ost02_106 D ffff9c8a393bc100 0 101338 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592597.912810] Call Trace: [Tue Mar 17 18:01:53 2020][8592597.915440] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:53 2020][8592597.921452] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592597.928094] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:53 2020][8592597.934453] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:53 2020][8592597.939802] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.946816] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.953898] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.961644] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:53 2020][8592597.968912] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.976774] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.983741] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:53 2020][8592597.989211] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592597.995685] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.003258] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592598.008335] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.014604] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592598.021224] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.027503] ll_ost00_088 S ffff9c6415de0000 0 101339 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.034862] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.037492] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592598.043593] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.048791] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.055666] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592598.061450] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.067937] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.075505] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592598.080565] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.086848] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592598.093461] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.099727] ll_ost02_107 S ffff9c6415de1040 0 101340 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.107101] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.109729] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592598.115821] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.121019] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.127895] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592598.133678] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.140169] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.147740] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592598.152793] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.159074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592598.165691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.171965] ll_ost03_067 S ffff9c7d5df72080 0 998 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.179251] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.181878] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592598.187988] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.193158] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.200036] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592598.205836] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.212313] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.219882] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592598.224950] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.231220] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592598.237855] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.244122] ll_ost03_068 S ffff9c69b7076180 0 55698 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.251391] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.254035] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592598.260132] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.265324] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.272204] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:53 2020][8592598.277989] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.284478] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.292067] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:53 2020][8592598.297122] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.303408] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:53 2020][8592598.310025] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:53 2020][8592598.316291] crond S ffff9c790c524100 0 57399 50991 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.323576] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.326206] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.331347] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:01:53 2020][8592598.336595] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:53 2020][8592598.342604] [] pipe_read+0x298/0x400 [Tue Mar 17 18:01:53 2020][8592598.348015] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:01:53 2020][8592598.353535] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:01:53 2020][8592598.358767] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:01:53 2020][8592598.363921] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:01:53 2020][8592598.370124] raid-check S ffff9c7a27d40000 0 57403 57399 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.377416] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.380045] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.385194] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:01:53 2020][8592598.390440] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:01:53 2020][8592598.395758] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:01:53 2020][8592598.401859] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:01:53 2020][8592598.408062] ll_ost00_089 S ffff9c68c37c6180 0 70314 2 0x00000080 [Tue Mar 17 18:01:53 2020][8592598.415331] Call Trace: [Tue Mar 17 18:01:53 2020][8592598.417960] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:53 2020][8592598.424053] [] schedule+0x29/0x70 [Tue Mar 17 18:01:53 2020][8592598.429238] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:53 2020][8592598.436119] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:54 2020][8592598.441902] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.448391] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.455965] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:54 2020][8592598.461016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.467331] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:54 2020][8592598.473948] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.480225] ll_ost03_069 D ffff9c79ca36d140 0 70315 2 0x00000080 [Tue Mar 17 18:01:54 2020][8592598.487517] Call Trace: [Tue Mar 17 18:01:54 2020][8592598.490146] [] schedule+0x29/0x70 [Tue Mar 17 18:01:54 2020][8592598.495294] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.502357] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:54 2020][8592598.508366] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.515609] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.522316] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.529810] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:54 2020][8592598.535996] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.542809] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.550128] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.557700] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.564853] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:54 2020][8592598.571121] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:54 2020][8592598.577225] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:54 2020][8592598.583971] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.591754] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.599759] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:54 2020][8592598.606249] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.613332] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.621080] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:54 2020][8592598.628345] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.636202] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.643179] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:54 2020][8592598.648959] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.655433] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.663032] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:54 2020][8592598.668092] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.674384] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:54 2020][8592598.681005] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.687280] ll_ost03_070 D ffff9c79ca36c100 0 70316 2 0x00000080 [Tue Mar 17 18:01:54 2020][8592598.694564] Call Trace: [Tue Mar 17 18:01:54 2020][8592598.697196] [] schedule+0x29/0x70 [Tue Mar 17 18:01:54 2020][8592598.702346] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.709410] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:54 2020][8592598.715422] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.722681] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.729391] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.736887] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:54 2020][8592598.743072] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.749867] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.757199] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.764777] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.771915] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:54 2020][8592598.778188] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:54 2020][8592598.784291] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:54 2020][8592598.791048] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.798822] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.806833] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:54 2020][8592598.813344] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.820426] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.828175] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:54 2020][8592598.835436] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.843297] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.850268] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:54 2020][8592598.856055] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.862524] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592598.870112] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:54 2020][8592598.875167] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.881441] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:54 2020][8592598.888078] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.894347] ll_ost03_071 D ffff9c79ca369040 0 70317 2 0x00000080 [Tue Mar 17 18:01:54 2020][8592598.901624] Call Trace: [Tue Mar 17 18:01:54 2020][8592598.904251] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592598.909577] [] schedule+0x29/0x70 [Tue Mar 17 18:01:54 2020][8592598.914720] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:54 2020][8592598.920728] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592598.926046] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:54 2020][8592598.932229] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:54 2020][8592598.937635] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:54 2020][8592598.943036] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:01:54 2020][8592598.949050] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592598.954371] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:01:54 2020][8592598.961254] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592598.967356] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:01:54 2020][8592598.972964] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.979868] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592598.988131] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:54 2020][8592598.995726] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.003993] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.010976] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.018728] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.025623] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.032764] [] evict+0xb4/0x180 [Tue Mar 17 18:01:54 2020][8592599.037736] [] iput+0xfc/0x190 [Tue Mar 17 18:01:54 2020][8592599.042640] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.049993] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:01:54 2020][8592599.057401] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:01:54 2020][8592599.064566] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:01:54 2020][8592599.071184] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:01:54 2020][8592599.077938] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.085707] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.093724] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:54 2020][8592599.100209] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.107298] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.115060] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:54 2020][8592599.122329] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.130187] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.137167] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:54 2020][8592599.142598] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.149086] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.156653] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:54 2020][8592599.161729] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592599.167997] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:54 2020][8592599.174609] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592599.180890] ll_ost03_072 S ffff9c79e44e9040 0 70318 2 0x00000080 [Tue Mar 17 18:01:54 2020][8592599.188179] Call Trace: [Tue Mar 17 18:01:54 2020][8592599.190812] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:54 2020][8592599.196908] [] schedule+0x29/0x70 [Tue Mar 17 18:01:54 2020][8592599.202093] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.208973] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:54 2020][8592599.214759] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.221247] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:54 2020][8592599.228821] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:54 2020][8592599.233897] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592599.240165] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:54 2020][8592599.246786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592599.253080] ll_ost03_073 D ffff9c79e44eb0c0 0 70319 2 0x00000080 [Tue Mar 17 18:01:54 2020][8592599.260357] Call Trace: [Tue Mar 17 18:01:54 2020][8592599.262984] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:54 2020][8592599.268299] [] ? bh_lru_install+0x18a/0x1e0 [Tue Mar 17 18:01:54 2020][8592599.274324] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592599.276301] LNet: Service thread pid 68338 was inactive for 200.20s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:01:54 2020][8592599.276303] LNet: Skipped 212 previous similar messages [Tue Mar 17 18:01:54 2020][8592599.276306] LustreError: dumping log to /tmp/lustre-log.1584493314.68338 [Tue Mar 17 18:01:54 2020][8592599.304974] [] schedule+0x29/0x70 [Tue Mar 17 18:01:54 2020][8592599.310116] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:54 2020][8592599.316129] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592599.321465] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:54 2020][8592599.327646] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:54 2020][8592599.333060] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:54 2020][8592599.338463] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:54 2020][8592599.344045] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:54 2020][8592599.349380] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:54 2020][8592599.355823] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:54 2020][8592599.361928] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:54 2020][8592599.368842] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.377131] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:54 2020][8592599.384721] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.392991] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.399962] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.407724] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:01:54 2020][8592599.414606] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.421758] [] evict+0xb4/0x180 [Tue Mar 17 18:01:55 2020][8592599.426725] [] iput+0xfc/0x190 [Tue Mar 17 18:01:55 2020][8592599.431620] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.438974] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:01:55 2020][8592599.446379] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:01:55 2020][8592599.453548] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:01:55 2020][8592599.460166] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:01:55 2020][8592599.466911] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.474691] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.482712] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:55 2020][8592599.489210] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.496291] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.504058] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:55 2020][8592599.511323] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.519184] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.526163] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:55 2020][8592599.531598] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.538083] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.545651] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592599.550725] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.556994] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592599.563608] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.569900] ll_ost03_074 D ffff9c743d2a1040 0 70320 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592599.577194] Call Trace: [Tue Mar 17 18:01:55 2020][8592599.579824] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592599.584976] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.592040] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:55 2020][8592599.598050] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.605301] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.612013] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.619517] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:55 2020][8592599.625708] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.632506] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.639849] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.647429] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.654583] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:55 2020][8592599.660860] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:55 2020][8592599.666962] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:55 2020][8592599.673707] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.681475] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.689487] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:55 2020][8592599.695977] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.703062] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.710837] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:55 2020][8592599.718085] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.725961] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.732920] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:55 2020][8592599.738348] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.744838] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.752411] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592599.757479] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.763748] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592599.770370] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.776673] ll_ost03_075 D ffff9c743d2a0000 0 70321 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592599.783947] Call Trace: [Tue Mar 17 18:01:55 2020][8592599.786577] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592599.791728] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.798793] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:55 2020][8592599.804805] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.812047] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.818757] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.826259] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:55 2020][8592599.832442] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:55 2020][8592599.839240] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.846586] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.854166] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:55 2020][8592599.861305] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:55 2020][8592599.867593] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:55 2020][8592599.873698] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:55 2020][8592599.880444] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.888209] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.896215] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:55 2020][8592599.902706] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.909802] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.917554] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:55 2020][8592599.924819] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.932684] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.939662] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:55 2020][8592599.945092] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.951582] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592599.959155] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592599.964224] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.970492] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592599.977105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592599.983398] ll_ost03_076 D ffff9c743d2a30c0 0 70322 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592599.990666] Call Trace: [Tue Mar 17 18:01:55 2020][8592599.993298] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592599.999918] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:55 2020][8592600.006299] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:55 2020][8592600.011650] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.018658] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.025742] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.033501] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:55 2020][8592600.040756] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.048648] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.055609] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:55 2020][8592600.061045] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.067535] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.075109] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592600.080184] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.086452] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592600.093065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.099345] ll_ost03_077 S ffff9c743d2a6180 0 70324 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592600.106635] Call Trace: [Tue Mar 17 18:01:55 2020][8592600.109266] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:55 2020][8592600.115380] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592600.120550] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.127430] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:55 2020][8592600.133229] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.139704] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.147278] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592600.152350] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.158620] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592600.165234] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.171529] ll_ost03_078 S ffff9c743d2a5140 0 70325 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592600.178803] Call Trace: [Tue Mar 17 18:01:55 2020][8592600.181431] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:55 2020][8592600.187527] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592600.192716] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.199590] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:55 2020][8592600.205387] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.211855] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.219441] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592600.224496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.230793] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592600.237412] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.243684] ll_ost03_079 S ffff9c73128630c0 0 70326 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592600.250972] Call Trace: [Tue Mar 17 18:01:55 2020][8592600.253602] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:55 2020][8592600.259702] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592600.264894] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.271774] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:55 2020][8592600.277549] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.284052] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.291622] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592600.296683] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.302962] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592600.309577] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.315846] ll_ost03_080 S ffff9c7312865140 0 70327 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592600.323138] Call Trace: [Tue Mar 17 18:01:55 2020][8592600.325767] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:55 2020][8592600.331860] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592600.337045] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.343939] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:55 2020][8592600.349718] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.356207] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.363781] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:55 2020][8592600.368844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.375132] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:55 2020][8592600.381748] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:55 2020][8592600.388037] ll_ost03_081 S ffff9c7312862080 0 70328 2 0x00000080 [Tue Mar 17 18:01:55 2020][8592600.395307] Call Trace: [Tue Mar 17 18:01:55 2020][8592600.397935] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:55 2020][8592600.404029] [] schedule+0x29/0x70 [Tue Mar 17 18:01:55 2020][8592600.409235] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:55 2020][8592600.416111] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592600.421893] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.428368] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.435941] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592600.441016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.447285] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592600.453906] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.460194] ll_ost03_082 D ffff9c5a39ea9040 0 70329 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592600.467483] Call Trace: [Tue Mar 17 18:01:56 2020][8592600.470111] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592600.475284] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.482345] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:56 2020][8592600.488358] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.495600] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.502309] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.509803] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:56 2020][8592600.515988] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.522785] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.530133] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.537710] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.544864] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:56 2020][8592600.551143] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:56 2020][8592600.557243] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:56 2020][8592600.563987] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.571756] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.579783] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:56 2020][8592600.586262] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.593358] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.601106] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:56 2020][8592600.608386] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.616244] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.623226] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:56 2020][8592600.628656] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.635131] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.642715] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592600.647771] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.654053] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592600.660668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.666943] ll_ost03_083 S ffff9c6ae99d30c0 0 70330 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592600.674237] Call Trace: [Tue Mar 17 18:01:56 2020][8592600.676867] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:56 2020][8592600.682984] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592600.688153] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.695032] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592600.700829] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.707298] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.714868] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592600.719937] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.726208] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592600.732856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.739129] ll_ost03_084 D ffff9c72ca02e180 0 70523 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592600.746423] Call Trace: [Tue Mar 17 18:01:56 2020][8592600.749050] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592600.754194] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.761259] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:56 2020][8592600.767271] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.774513] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.781222] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.788737] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:56 2020][8592600.794927] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:56 2020][8592600.801724] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.809057] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.816631] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:56 2020][8592600.823787] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:56 2020][8592600.830064] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:56 2020][8592600.836164] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:56 2020][8592600.842894] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.850669] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.858709] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:56 2020][8592600.865183] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.872278] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.880028] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:56 2020][8592600.887279] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.895156] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.902123] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592600.907907] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.914395] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.921971] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592600.927059] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.933331] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592600.939944] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592600.946233] ll_ost03_085 D ffff9c72ca02b0c0 0 70524 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592600.953506] Call Trace: [Tue Mar 17 18:01:56 2020][8592600.956135] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:01:56 2020][8592600.962148] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592600.968778] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:56 2020][8592600.975133] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:56 2020][8592600.980486] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.987511] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592600.994592] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.002357] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:56 2020][8592601.009611] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.017470] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.024450] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:56 2020][8592601.029881] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.036371] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.043945] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592601.048998] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.055280] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592601.061894] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.068160] ll_ost03_086 S ffff9c72ca029040 0 70525 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592601.075437] Call Trace: [Tue Mar 17 18:01:56 2020][8592601.078097] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 18:01:56 2020][8592601.085408] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592601.090568] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 18:01:56 2020][8592601.096582] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:01:56 2020][8592601.103148] [] ldlm_completion_ast+0x430/0x860 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.110243] [] ? ldlm_completion_ast_async+0x300/0x300 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.118009] [] ? ldlm_lock_enqueue+0x3ff/0xa20 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.125087] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592601.130863] [] ldlm_cli_enqueue_local+0x231/0x830 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.138197] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.146248] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.154001] [] ofd_destroy_by_fid+0x1dd/0x4a0 [ofd] [Tue Mar 17 18:01:56 2020][8592601.160745] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.168513] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.176524] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:56 2020][8592601.183020] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.190105] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.197884] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:56 2020][8592601.205140] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.213002] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.219984] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:56 2020][8592601.225421] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.231908] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.239474] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592601.244542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.250810] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592601.257425] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.263726] ll_ost03_087 S ffff9c896f8dd140 0 71193 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592601.271001] Call Trace: [Tue Mar 17 18:01:56 2020][8592601.273632] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:56 2020][8592601.279733] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592601.284922] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.291798] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:56 2020][8592601.298416] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592601.304207] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.310675] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.318246] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592601.323316] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.324308] LustreError: dumping log to /tmp/lustre-log.1584493316.70523 [Tue Mar 17 18:01:56 2020][8592601.336470] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592601.343100] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.349369] ll_ost03_088 S ffff9c896f8de180 0 71194 2 0x00000080 [Tue Mar 17 18:01:56 2020][8592601.356659] Call Trace: [Tue Mar 17 18:01:56 2020][8592601.359299] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:56 2020][8592601.365411] [] schedule+0x29/0x70 [Tue Mar 17 18:01:56 2020][8592601.370580] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.377459] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:56 2020][8592601.383255] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.389724] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:56 2020][8592601.397301] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:56 2020][8592601.402373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:56 2020][8592601.408642] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:56 2020][8592601.415276] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.421546] ll_ost03_089 D ffff9c896f8db0c0 0 71195 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592601.428824] Call Trace: [Tue Mar 17 18:01:57 2020][8592601.431465] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592601.438081] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:01:57 2020][8592601.444464] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:01:57 2020][8592601.449786] [] dqget+0x19f/0x450 [Tue Mar 17 18:01:57 2020][8592601.454859] [] dquot_get_dqblk+0x14/0x1f0 [Tue Mar 17 18:01:57 2020][8592601.460707] [] osd_acct_index_lookup+0x235/0x480 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.468388] [] lquotactl_slv+0x27d/0x9d0 [lquota] [Tue Mar 17 18:01:57 2020][8592601.474935] [] ofd_quotactl+0x13c/0x380 [ofd] [Tue Mar 17 18:01:57 2020][8592601.481154] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.488248] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.495998] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:57 2020][8592601.503277] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.511136] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.518118] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:57 2020][8592601.523547] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.530022] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.537606] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592601.542662] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.548945] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592601.555559] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.561837] ll_ost03_090 D ffff9c5ae95a1040 0 71243 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592601.569137] Call Trace: [Tue Mar 17 18:01:57 2020][8592601.571766] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592601.576919] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.583984] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:57 2020][8592601.589995] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.597237] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.603945] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.611442] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:57 2020][8592601.617625] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.624422] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.631772] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.639346] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.646486] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:57 2020][8592601.652776] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:57 2020][8592601.658880] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:57 2020][8592601.665610] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.673398] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.681404] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:57 2020][8592601.687895] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.694978] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.702751] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:57 2020][8592601.710004] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.717879] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.724837] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:57 2020][8592601.730267] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.736757] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.744328] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592601.749396] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.755665] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592601.762278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.768571] ll_ost03_091 D ffff9c790c525140 0 71289 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592601.775838] Call Trace: [Tue Mar 17 18:01:57 2020][8592601.778468] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592601.783620] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.790687] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:57 2020][8592601.796697] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.803940] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.810654] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.818150] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:57 2020][8592601.824334] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:57 2020][8592601.831134] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.838477] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.846056] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592601.853212] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:57 2020][8592601.859490] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:57 2020][8592601.865589] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:57 2020][8592601.872338] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.880112] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.888122] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:57 2020][8592601.894616] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.901697] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.909473] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:57 2020][8592601.916721] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.924582] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.931562] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:57 2020][8592601.936994] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.943467] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592601.951057] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592601.956120] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.962407] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592601.969038] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592601.975307] ll_ost03_092 S ffff9c7f8cf74100 0 74373 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592601.982593] Call Trace: [Tue Mar 17 18:01:57 2020][8592601.985221] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:57 2020][8592601.991312] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592601.996497] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.003377] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:57 2020][8592602.009154] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.015643] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.023217] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592602.028314] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.034589] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592602.041208] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.047497] ll_ost00_090 S ffff9c7f8cf76180 0 74374 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592602.054769] Call Trace: [Tue Mar 17 18:01:57 2020][8592602.057398] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:57 2020][8592602.063498] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592602.068681] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.075556] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:57 2020][8592602.081336] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.087842] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.095412] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592602.100487] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.106754] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592602.113376] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.119665] ll_ost03_093 D ffff9c81cd75d140 0 74613 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592602.126937] Call Trace: [Tue Mar 17 18:01:57 2020][8592602.129564] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592602.134706] [] __wait_on_freeing_inode+0xb0/0xf0 [Tue Mar 17 18:01:57 2020][8592602.141169] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.147276] [] find_inode_fast+0x71/0xb0 [Tue Mar 17 18:01:57 2020][8592602.153027] [] iget_locked+0x79/0x1a0 [Tue Mar 17 18:01:57 2020][8592602.158541] [] ldiskfs_iget+0x42/0xc50 [ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.164994] [] osd_iget+0x26/0x2d0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.171444] [] osd_obj_map_lookup+0x260/0x510 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.178866] [] ? osd_fld_lookup+0x48/0xd0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.185921] [] osd_oi_lookup+0x139/0x1e0 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.192891] [] osd_fid_lookup+0x4a5/0x1c60 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.200047] [] ? kiblnd_queue_tx+0x36/0x50 [ko2iblnd] [Tue Mar 17 18:01:57 2020][8592602.206928] [] ? kiblnd_launch_tx+0xd2/0xc20 [ko2iblnd] [Tue Mar 17 18:01:57 2020][8592602.214008] [] ? lprocfs_counter_add+0xf9/0x160 [obdclass] [Tue Mar 17 18:01:57 2020][8592602.221352] [] osd_object_init+0x61/0x110 [osd_ldiskfs] [Tue Mar 17 18:01:57 2020][8592602.228420] [] lu_object_start.isra.35+0x8b/0x120 [obdclass] [Tue Mar 17 18:01:57 2020][8592602.235925] [] lu_object_find_at+0x1e1/0xa60 [obdclass] [Tue Mar 17 18:01:57 2020][8592602.242999] [] ? cfs_hash_bd_lookup_intent+0x38/0x170 [libcfs] [Tue Mar 17 18:01:57 2020][8592602.250675] [] lu_object_find+0x16/0x20 [obdclass] [Tue Mar 17 18:01:57 2020][8592602.257305] [] ofd_object_find+0x35/0x100 [ofd] [Tue Mar 17 18:01:57 2020][8592602.263685] [] ofd_lvbo_init+0x363/0x7f0 [ofd] [Tue Mar 17 18:01:57 2020][8592602.269986] [] ? ldlm_extent_alloc_lock+0xb0/0x280 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.277411] [] ? ldlm_lock_create+0x4f8/0x9f0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.284427] [] ldlm_handle_enqueue0+0x97c/0x15f0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.291679] [] ? lustre_swab_ldlm_lock_desc+0x30/0x30 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.299386] [] tgt_enqueue+0x62/0x210 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.305689] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.312767] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.320530] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:57 2020][8592602.327784] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.335646] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.342624] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:57 2020][8592602.348056] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.348311] LustreError: dumping log to /tmp/lustre-log.1584493317.98687 [Tue Mar 17 18:01:57 2020][8592602.361464] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.369050] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:57 2020][8592602.374105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.380388] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:57 2020][8592602.387002] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:57 2020][8592602.393267] ll_ost03_094 S ffff9c84b1d15140 0 74614 2 0x00000080 [Tue Mar 17 18:01:57 2020][8592602.400555] Call Trace: [Tue Mar 17 18:01:57 2020][8592602.403183] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:57 2020][8592602.409304] [] schedule+0x29/0x70 [Tue Mar 17 18:01:57 2020][8592602.414494] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:57 2020][8592602.421376] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592602.427172] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.433640] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.441213] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592602.446289] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.452556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592602.459171] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.465457] ll_ost00_091 S ffff9c84b1d10000 0 74615 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592602.472748] Call Trace: [Tue Mar 17 18:01:58 2020][8592602.475377] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592602.481478] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592602.486684] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.493560] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592602.499342] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.505830] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.513399] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592602.518475] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.524743] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592602.531364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.537665] ll_ost03_095 D ffff9c84b1d16180 0 74616 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592602.544943] Call Trace: [Tue Mar 17 18:01:58 2020][8592602.547586] [] ? __brelse+0x3d/0x50 [Tue Mar 17 18:01:58 2020][8592602.552901] [] ? bh_lru_install+0x18a/0x1e0 [Tue Mar 17 18:01:58 2020][8592602.558931] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:58 2020][8592602.564245] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592602.569391] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:01:58 2020][8592602.575414] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:58 2020][8592602.580729] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:01:58 2020][8592602.586909] [] io_schedule+0x18/0x20 [Tue Mar 17 18:01:58 2020][8592602.592344] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:01:58 2020][8592602.597751] [] __wait_on_bit+0x67/0x90 [Tue Mar 17 18:01:58 2020][8592602.603323] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:01:58 2020][8592602.608651] [] out_of_line_wait_on_bit+0x81/0xb0 [Tue Mar 17 18:01:58 2020][8592602.615094] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.621199] [] do_get_write_access+0x285/0x4d0 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.628098] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.636365] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.643958] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.652228] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.659226] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.666977] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.673876] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.681013] [] evict+0xb4/0x180 [Tue Mar 17 18:01:58 2020][8592602.685986] [] iput+0xfc/0x190 [Tue Mar 17 18:01:58 2020][8592602.690885] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.698219] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:01:58 2020][8592602.705623] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:01:58 2020][8592602.712791] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:01:58 2020][8592602.719411] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:01:58 2020][8592602.726141] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.733943] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.741952] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:58 2020][8592602.748443] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.755527] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.763272] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:58 2020][8592602.770539] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.778394] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.785360] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:58 2020][8592602.790811] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.797304] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.804874] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592602.809944] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.816213] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592602.822849] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.829119] ll_ost03_096 S ffff9c84b1d130c0 0 74617 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592602.836395] Call Trace: [Tue Mar 17 18:01:58 2020][8592602.839038] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592602.845133] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592602.850321] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.857212] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592602.863004] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.869474] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592602.877061] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592602.882115] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.888404] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592602.895019] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592602.901304] ll_ost03_097 D ffff9c896f8dc100 0 74618 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592602.908579] Call Trace: [Tue Mar 17 18:01:58 2020][8592602.911211] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592602.916361] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.923441] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:58 2020][8592602.929457] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.936697] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.943406] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.950894] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:58 2020][8592602.957099] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:58 2020][8592602.963889] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.971208] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.978795] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:58 2020][8592602.985952] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:58 2020][8592602.992227] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:58 2020][8592602.998347] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:58 2020][8592603.005081] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.012852] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.020870] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:58 2020][8592603.027350] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.034430] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.042192] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:58 2020][8592603.049447] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.057343] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.064309] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592603.070105] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.076583] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.084154] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592603.089222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.095491] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592603.102112] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.108398] ll_ost03_098 S ffff9c86ccb58000 0 78398 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592603.115673] Call Trace: [Tue Mar 17 18:01:58 2020][8592603.118303] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592603.124403] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592603.129607] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.136483] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592603.142258] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.148746] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.156313] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592603.161364] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.167646] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592603.174261] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.180543] ll_ost00_092 S ffff9c7ae99f2080 0 85257 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592603.187813] Call Trace: [Tue Mar 17 18:01:58 2020][8592603.190445] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592603.196543] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592603.201727] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.208609] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592603.214408] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.220881] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.228456] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592603.233530] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.239800] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592603.246437] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.252705] ll_ost00_093 S ffff9c8a393a1040 0 85258 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592603.259980] Call Trace: [Tue Mar 17 18:01:58 2020][8592603.262624] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592603.268721] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592603.273911] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.280788] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:58 2020][8592603.287421] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592603.293200] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.299671] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.307260] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592603.312315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.318603] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592603.325218] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.331509] ll_ost00_094 S ffff9c8a35310000 0 85259 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592603.338779] Call Trace: [Tue Mar 17 18:01:58 2020][8592603.341408] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592603.347501] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592603.352691] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.359598] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:58 2020][8592603.365375] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.371863] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:58 2020][8592603.379439] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:58 2020][8592603.384499] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.390789] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:58 2020][8592603.397402] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:58 2020][8592603.403670] ll_ost00_095 S ffff9c7c83ea4100 0 85260 2 0x00000080 [Tue Mar 17 18:01:58 2020][8592603.410957] Call Trace: [Tue Mar 17 18:01:58 2020][8592603.413587] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:58 2020][8592603.419687] [] schedule+0x29/0x70 [Tue Mar 17 18:01:58 2020][8592603.424889] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.431768] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592603.437553] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.444038] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.451604] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.456674] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.462941] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.469564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.475868] ll_ost02_108 S ffff9c7c83ea30c0 0 85261 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.483157] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.485788] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592603.491890] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.497079] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.503954] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592603.509736] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.516224] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.523790] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.528844] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.535133] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.541749] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.548036] ll_ost00_096 S ffff9c7c83ea6180 0 85262 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.555309] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.557939] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592603.564053] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.569225] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.576103] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592603.581892] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.588362] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.595933] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.601023] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.607295] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.613917] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.620209] ll_ost02_109 D ffff9c7c83ea5140 0 85263 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.627485] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.630114] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.635257] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:59 2020][8592603.642323] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:59 2020][8592603.648333] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:59 2020][8592603.655577] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:59 2020][8592603.662293] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592603.669803] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:59 2020][8592603.675989] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:59 2020][8592603.682788] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592603.690118] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:59 2020][8592603.697694] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592603.704834] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:59 2020][8592603.711122] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:59 2020][8592603.717222] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:59 2020][8592603.723951] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.731751] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.739759] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:59 2020][8592603.746239] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.753340] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.761092] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:59 2020][8592603.768357] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.776212] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.783194] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:59 2020][8592603.788622] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.795114] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.802701] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.807758] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.814046] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.820660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.826928] ll_ost00_097 S ffff9c81cd75c100 0 85264 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.834220] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.836851] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592603.842952] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.848145] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.855025] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:59 2020][8592603.861645] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592603.867448] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.873920] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.881491] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.886562] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.892828] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.899441] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.905723] ll_ost02_110 S ffff9c81cd75a080 0 85265 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.912992] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.915623] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.920799] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.927707] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592603.933487] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.939976] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592603.947551] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592603.952612] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.958900] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592603.965517] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592603.971805] ll_ost02_111 D ffff9c81cd75e180 0 85266 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592603.979078] Call Trace: [Tue Mar 17 18:01:59 2020][8592603.981705] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592603.986858] [] wait_transaction_locked+0x85/0xd0 [jbd2] [Tue Mar 17 18:01:59 2020][8592603.993936] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:01:59 2020][8592603.999953] [] add_transaction_credits+0x268/0x2f0 [jbd2] [Tue Mar 17 18:01:59 2020][8592604.007196] [] start_this_handle+0x1a1/0x430 [jbd2] [Tue Mar 17 18:01:59 2020][8592604.013911] [] ? osd_declare_write+0x350/0x490 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592604.021391] [] ? kmem_cache_alloc+0x1c2/0x1f0 [Tue Mar 17 18:01:59 2020][8592604.027588] [] jbd2__journal_start+0xf3/0x1f0 [jbd2] [Tue Mar 17 18:01:59 2020][8592604.034388] [] ? osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592604.041705] [] __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:01:59 2020][8592604.049307] [] osd_trans_start+0x20e/0x4e0 [osd_ldiskfs] [Tue Mar 17 18:01:59 2020][8592604.056453] [] ofd_trans_start+0x75/0xf0 [ofd] [Tue Mar 17 18:01:59 2020][8592604.062741] [] ofd_destroy+0x5d0/0x960 [ofd] [Tue Mar 17 18:01:59 2020][8592604.068845] [] ofd_destroy_by_fid+0x1f4/0x4a0 [ofd] [Tue Mar 17 18:01:59 2020][8592604.075576] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.083364] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.091371] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:01:59 2020][8592604.097847] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.104941] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.112691] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:01:59 2020][8592604.119973] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.127829] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.134810] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:01:59 2020][8592604.140241] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.146716] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.154307] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592604.159365] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.165647] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592604.172262] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.178535] ll_ost00_098 S ffff9c8a2e1d8000 0 85267 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592604.185829] Call Trace: [Tue Mar 17 18:01:59 2020][8592604.188459] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592604.194567] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592604.199738] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.206617] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592604.212414] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.218893] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.226462] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592604.231528] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.237799] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592604.244421] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.250724] ll_ost00_099 S ffff9c836cd00000 0 85268 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592604.257998] Call Trace: [Tue Mar 17 18:01:59 2020][8592604.260627] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592604.266729] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592604.271920] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.278802] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:01:59 2020][8592604.285423] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592604.291223] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.297699] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.305270] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592604.310344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.316615] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592604.323236] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.329525] ll_ost00_100 S ffff9c836cd01040 0 85269 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592604.336795] Call Trace: [Tue Mar 17 18:01:59 2020][8592604.339424] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592604.345517] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592604.350701] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.357602] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:01:59 2020][8592604.363383] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.369872] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:01:59 2020][8592604.377447] [] kthread+0xd1/0xe0 [Tue Mar 17 18:01:59 2020][8592604.382508] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.388789] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:01:59 2020][8592604.395405] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:01:59 2020][8592604.401677] ll_ost00_101 S ffff9c80cf9ae180 0 85270 2 0x00000080 [Tue Mar 17 18:01:59 2020][8592604.408964] Call Trace: [Tue Mar 17 18:01:59 2020][8592604.411593] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:01:59 2020][8592604.417695] [] schedule+0x29/0x70 [Tue Mar 17 18:01:59 2020][8592604.422903] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.429778] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.435560] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.442047] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.449614] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.454675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.460957] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.467571] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.473851] ll_ost00_102 S ffff9c86ccb5d140 0 85271 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.481142] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.483769] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592604.489863] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.495067] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.501945] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.507744] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.514217] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.521790] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.526858] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.533125] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.539738] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.546036] ll_ost00_103 D ffff9c86ccb5a080 0 85272 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.553310] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.555938] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.562557] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:02:00 2020][8592604.568926] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:02:00 2020][8592604.574277] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.581279] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.588371] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.596122] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:00 2020][8592604.603389] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.611265] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.618226] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:00 2020][8592604.623677] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.630154] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.637725] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.642792] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.649061] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.655676] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.661963] ll_ost00_104 S ffff9c86ccb5c100 0 85273 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.669233] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.671877] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592604.677973] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.683181] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.690056] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.695839] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.702328] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.709901] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.714968] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.721238] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.727875] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.734158] ll_ost00_105 S ffff9c86ccb5b0c0 0 85274 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.741428] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.744073] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592604.750168] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.755358] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.762234] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:02:00 2020][8592604.768854] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.774648] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.781118] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.788691] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.793780] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.800055] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.806675] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.812965] ll_ost00_106 S ffff9c8a1f3ce180 0 85275 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.820235] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.822865] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592604.828965] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.834148] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.841028] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.846804] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.853311] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.860883] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.865952] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.872221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.878848] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.885117] ll_ost00_107 S ffff9c870e4e8000 0 85276 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.892385] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.895027] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592604.901125] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.906318] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.913212] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592604.919005] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.925481] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.933066] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592604.938122] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.944406] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592604.951018] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592604.957284] ll_ost00_108 D ffff9c870e4e9040 0 85277 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592604.964570] Call Trace: [Tue Mar 17 18:02:00 2020][8592604.967199] [] ? __enqueue_entity+0x78/0x80 [Tue Mar 17 18:02:00 2020][8592604.973206] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592604.979856] [] __mutex_lock_slowpath+0xc7/0x1d0 [Tue Mar 17 18:02:00 2020][8592604.986216] [] mutex_lock+0x1f/0x2f [Tue Mar 17 18:02:00 2020][8592604.991571] [] req_can_reconstruct+0x42/0x100 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592604.998587] [] tgt_request_handle+0xdba/0x1580 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.005668] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.013417] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:00 2020][8592605.020684] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.028546] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.035525] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:00 2020][8592605.040955] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.047460] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.055028] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592605.060095] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592605.066365] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592605.073001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592605.079268] ll_ost03_099 D ffff9c523155c100 0 90274 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592605.086537] Call Trace: [Tue Mar 17 18:02:00 2020][8592605.089166] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:00 2020][8592605.094492] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592605.099649] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:02:00 2020][8592605.105663] [] ? __jbd2_journal_file_buffer+0xe4/0x220 [jbd2] [Tue Mar 17 18:02:00 2020][8592605.113247] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:00 2020][8592605.118566] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:02:00 2020][8592605.124766] [] io_schedule+0x18/0x20 [Tue Mar 17 18:02:00 2020][8592605.130169] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:02:00 2020][8592605.135577] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:02:00 2020][8592605.141605] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:00 2020][8592605.146920] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:02:00 2020][8592605.153795] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592605.159923] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:02:00 2020][8592605.165505] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:02:00 2020][8592605.172406] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.180674] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:02:00 2020][8592605.188265] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.196536] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.203519] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.211270] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.218152] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.225304] [] evict+0xb4/0x180 [Tue Mar 17 18:02:00 2020][8592605.230292] [] iput+0xfc/0x190 [Tue Mar 17 18:02:00 2020][8592605.235182] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:02:00 2020][8592605.242525] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:02:00 2020][8592605.249933] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:02:00 2020][8592605.257084] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:02:00 2020][8592605.263718] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:02:00 2020][8592605.270449] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.278223] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.286252] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:02:00 2020][8592605.292729] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.299808] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.307584] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:00 2020][8592605.314833] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.322694] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.329673] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:00 2020][8592605.335107] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.341579] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.349166] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:00 2020][8592605.354222] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592605.360511] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:00 2020][8592605.367140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:00 2020][8592605.373409] ll_ost03_100 S ffff9c523155b0c0 0 90275 2 0x00000080 [Tue Mar 17 18:02:00 2020][8592605.380694] Call Trace: [Tue Mar 17 18:02:00 2020][8592605.383324] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:00 2020][8592605.389417] [] schedule+0x29/0x70 [Tue Mar 17 18:02:00 2020][8592605.394608] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:00 2020][8592605.401490] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:00 2020][8592605.407272] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.413765] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.421336] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592605.426389] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.432684] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592605.439303] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.445575] ll_ost03_101 S ffff9c523155a080 0 90276 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592605.452864] Call Trace: [Tue Mar 17 18:02:01 2020][8592605.455492] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592605.461593] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592605.466784] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.473673] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592605.479455] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.485938] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.493528] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592605.498584] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.504889] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592605.511506] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.517809] ll_ost03_102 D ffff9c6ec59fc100 0 112801 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592605.525169] Call Trace: [Tue Mar 17 18:02:01 2020][8592605.527799] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:01 2020][8592605.533132] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592605.538276] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:02:01 2020][8592605.544285] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:01 2020][8592605.549630] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:02:01 2020][8592605.555817] [] io_schedule+0x18/0x20 [Tue Mar 17 18:02:01 2020][8592605.561230] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:02:01 2020][8592605.566633] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:02:01 2020][8592605.572641] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:01 2020][8592605.577977] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:02:01 2020][8592605.584851] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.590947] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:02:01 2020][8592605.596548] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:02:01 2020][8592605.603434] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.611731] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:02:01 2020][8592605.619311] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.627597] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.634571] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.642318] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.649213] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.656356] [] evict+0xb4/0x180 [Tue Mar 17 18:02:01 2020][8592605.661326] [] iput+0xfc/0x190 [Tue Mar 17 18:02:01 2020][8592605.666214] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:02:01 2020][8592605.673566] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:02:01 2020][8592605.680991] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:02:01 2020][8592605.688144] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:02:01 2020][8592605.694780] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:02:01 2020][8592605.701515] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.709290] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.717319] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:02:01 2020][8592605.723799] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.730898] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.738646] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:01 2020][8592605.745915] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.753776] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.760771] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592605.766553] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.773025] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.780614] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592605.785668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.791949] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592605.798564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.804832] ll_ost03_103 S ffff9c5ac1af1040 0 113486 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592605.812211] Call Trace: [Tue Mar 17 18:02:01 2020][8592605.814841] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592605.820942] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592605.826149] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.833025] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592605.838807] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.845296] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.852870] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592605.857931] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.864211] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592605.870827] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.877092] ll_ost00_109 S ffff9c8a31a75140 0 125734 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592605.884466] Call Trace: [Tue Mar 17 18:02:01 2020][8592605.887096] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592605.893195] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592605.898380] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.905261] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592605.911044] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.917532] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.925098] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592605.930151] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.936447] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592605.943065] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592605.949339] ll_ost00_110 S ffff9c7a324c8000 0 125735 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592605.956712] Call Trace: [Tue Mar 17 18:02:01 2020][8592605.959340] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592605.965433] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592605.970616] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.977489] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592605.983272] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.989763] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592605.997349] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.002406] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.008696] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.015315] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.021585] ll_ost00_111 S ffff9c7a324ce180 0 125736 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.028956] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.031586] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.037679] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.042864] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.049743] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592606.055518] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.062018] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.069590] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.074666] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.080936] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.087557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.093845] ll_ost03_104 S ffff9c72a7cd5140 0 125737 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.101204] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.103834] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.109933] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.115130] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.122007] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592606.127789] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.134280] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.141853] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.146907] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.153197] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.159811] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.166084] ll_ost03_105 S ffff9c72a7cd30c0 0 125738 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.173457] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.176086] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.182179] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.187375] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.194252] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592606.200028] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.206515] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.214080] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.219149] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.225417] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.232030] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.238317] ll_ost03_106 S ffff9c71fab9a080 0 125739 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.245679] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.248307] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.254407] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.259590] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.266464] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592606.272261] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.278730] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.286303] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.291377] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.297662] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.304275] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.310556] ll_ost03_107 S ffff9c71fab9b0c0 0 125740 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.317914] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.320544] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.326644] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.331830] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.338710] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:01 2020][8592606.344493] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.350981] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:01 2020][8592606.358548] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:01 2020][8592606.363637] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.369910] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:01 2020][8592606.376532] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:01 2020][8592606.382821] ll_ost03_108 S ffff9c752496c100 0 125741 2 0x00000080 [Tue Mar 17 18:02:01 2020][8592606.390178] Call Trace: [Tue Mar 17 18:02:01 2020][8592606.392808] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:01 2020][8592606.398910] [] schedule+0x29/0x70 [Tue Mar 17 18:02:01 2020][8592606.404101] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.410981] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:02 2020][8592606.416766] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.423266] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.430836] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592606.435905] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.442172] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592606.448786] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.455074] ll_ost03_109 D ffff9c752496a080 0 125742 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592606.462434] Call Trace: [Tue Mar 17 18:02:02 2020][8592606.465062] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.470384] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592606.475572] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:02:02 2020][8592606.481585] [] ? free_pcppages_bulk+0x17e/0x3a0 [Tue Mar 17 18:02:02 2020][8592606.487942] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.493277] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:02:02 2020][8592606.499460] [] io_schedule+0x18/0x20 [Tue Mar 17 18:02:02 2020][8592606.504858] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:02:02 2020][8592606.510273] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:02:02 2020][8592606.516282] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.521604] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:02:02 2020][8592606.528499] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.534611] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:02:02 2020][8592606.540191] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:02:02 2020][8592606.547089] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.555356] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:02:02 2020][8592606.562948] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.571217] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.578201] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.585951] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.592832] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.599983] [] evict+0xb4/0x180 [Tue Mar 17 18:02:02 2020][8592606.604952] [] iput+0xfc/0x190 [Tue Mar 17 18:02:02 2020][8592606.609874] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.617203] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:02:02 2020][8592606.624606] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:02:02 2020][8592606.631774] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:02:02 2020][8592606.638391] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:02:02 2020][8592606.645136] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.652905] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.660916] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:02:02 2020][8592606.667409] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.674489] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.682265] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:02 2020][8592606.689517] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.697390] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.704353] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:02:02 2020][8592606.710971] [] ? _raw_spin_lock+0x20/0x30 [Tue Mar 17 18:02:02 2020][8592606.716850] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.723329] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.730905] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592606.735971] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.742242] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592606.748891] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.755162] ll_ost03_110 D ffff9c752496d140 0 125743 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592606.762519] Call Trace: [Tue Mar 17 18:02:02 2020][8592606.765162] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.770477] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592606.775632] [] schedule_timeout+0x221/0x2d0 [Tue Mar 17 18:02:02 2020][8592606.781639] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.786952] [] io_schedule_timeout+0xad/0x130 [Tue Mar 17 18:02:02 2020][8592606.793146] [] io_schedule+0x18/0x20 [Tue Mar 17 18:02:02 2020][8592606.798563] [] bit_wait_io+0x11/0x50 [Tue Mar 17 18:02:02 2020][8592606.803965] [] __wait_on_bit_lock+0x61/0xc0 [Tue Mar 17 18:02:02 2020][8592606.809986] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:02 2020][8592606.815300] [] out_of_line_wait_on_bit_lock+0x81/0xb0 [Tue Mar 17 18:02:02 2020][8592606.822175] [] ? wake_bit_function+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592606.828289] [] __lock_buffer+0x32/0x40 [Tue Mar 17 18:02:02 2020][8592606.833870] [] do_get_write_access+0x432/0x4d0 [jbd2] [Tue Mar 17 18:02:02 2020][8592606.840754] [] ? __ldiskfs_handle_dirty_metadata+0x8b/0x220 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.849034] [] jbd2_journal_get_write_access+0x27/0x40 [jbd2] [Tue Mar 17 18:02:02 2020][8592606.856615] [] __ldiskfs_journal_get_write_access+0x3b/0x80 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.864880] [] ldiskfs_orphan_add+0xb9/0x240 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.871876] [] ? __ldiskfs_journal_start_sb+0x69/0xe0 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.879623] [] ldiskfs_truncate+0x1a7/0x3f0 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.886520] [] ldiskfs_evict_inode+0x58a/0x630 [ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.893659] [] evict+0xb4/0x180 [Tue Mar 17 18:02:02 2020][8592606.898632] [] iput+0xfc/0x190 [Tue Mar 17 18:02:02 2020][8592606.903539] [] osd_object_delete+0x1d2/0x330 [osd_ldiskfs] [Tue Mar 17 18:02:02 2020][8592606.910872] [] lu_object_free.isra.32+0x68/0x170 [obdclass] [Tue Mar 17 18:02:02 2020][8592606.918276] [] ? cfs_hash_bd_from_key+0x38/0xb0 [libcfs] [Tue Mar 17 18:02:02 2020][8592606.925443] [] lu_object_put+0xc5/0x3d0 [obdclass] [Tue Mar 17 18:02:02 2020][8592606.932064] [] ofd_destroy_by_fid+0x21a/0x4a0 [ofd] [Tue Mar 17 18:02:02 2020][8592606.938824] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.946593] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.954598] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:02:02 2020][8592606.961089] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.968169] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.975919] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:02 2020][8592606.983185] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.991048] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592606.998029] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:02 2020][8592607.003460] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.009962] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.017532] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592607.022608] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.028874] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592607.035489] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.041777] ll_ost03_111 S ffff9c752496b0c0 0 125744 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592607.049136] Call Trace: [Tue Mar 17 18:02:02 2020][8592607.051783] [] ? lprocfs_counter_sub+0xc1/0x130 [obdclass] [Tue Mar 17 18:02:02 2020][8592607.059097] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592607.064273] [] schedule_timeout+0x168/0x2d0 [Tue Mar 17 18:02:02 2020][8592607.070286] [] ? __internal_add_timer+0x130/0x130 [Tue Mar 17 18:02:02 2020][8592607.076846] [] ldlm_completion_ast+0x430/0x860 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.083933] [] ? ldlm_completion_ast_async+0x300/0x300 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.091700] [] ? ldlm_lock_enqueue+0x3ff/0xa20 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.098763] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:02 2020][8592607.104536] [] ldlm_cli_enqueue_local+0x231/0x830 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.111877] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.119928] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.127671] [] ofd_destroy_by_fid+0x1dd/0x4a0 [ofd] [Tue Mar 17 18:02:02 2020][8592607.134403] [] ? ldlm_blocking_ast_nocheck+0x310/0x310 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.142203] [] ? ldlm_expired_completion_wait+0x220/0x220 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.150216] [] ofd_destroy_hdl+0x267/0x970 [ofd] [Tue Mar 17 18:02:02 2020][8592607.156706] [] tgt_request_handle+0xaea/0x1580 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.163788] [] ? ptlrpc_nrs_req_get_nolock0+0xd1/0x170 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.171536] [] ? ktime_get_real_seconds+0xe/0x10 [libcfs] [Tue Mar 17 18:02:02 2020][8592607.178800] [] ptlrpc_server_handle_request+0x24b/0xab0 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.186656] [] ? ptlrpc_wait_event+0xa5/0x360 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.193622] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:02 2020][8592607.199075] [] ptlrpc_main+0xb2c/0x1460 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.205565] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.213139] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592607.218214] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.224483] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592607.231098] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.237385] kworker/14:2H S ffff9c6fb712c100 0 40930 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592607.244664] Call Trace: [Tue Mar 17 18:02:02 2020][8592607.247298] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:02 2020][8592607.253484] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592607.258645] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:02 2020][8592607.264394] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:02 2020][8592607.271123] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592607.276180] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.282447] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592607.289083] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.295352] kworker/27:1 S ffff9c8a2d8b8000 0 80638 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592607.302634] Call Trace: [Tue Mar 17 18:02:02 2020][8592607.305267] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:02 2020][8592607.311468] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592607.316611] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:02 2020][8592607.322357] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:02 2020][8592607.329080] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:02 2020][8592607.334135] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.340400] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:02 2020][8592607.347029] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:02 2020][8592607.353296] ll_ost_create03 S ffff9c7a27d44100 0 6305 2 0x00000080 [Tue Mar 17 18:02:02 2020][8592607.360565] Call Trace: [Tue Mar 17 18:02:02 2020][8592607.363195] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:02 2020][8592607.369334] [] schedule+0x29/0x70 [Tue Mar 17 18:02:02 2020][8592607.374510] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.381389] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:02:02 2020][8592607.388022] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:02 2020][8592607.393800] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:02 2020][8592607.400260] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:02:02 2020][8592607.405950] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592607.413537] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.418592] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.424858] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.431486] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.437754] ldlm_cb00_002 S ffff9c8a31a76180 0 6628 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.445032] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.447687] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:03 2020][8592607.453789] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.458978] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592607.465851] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:03 2020][8592607.468330] LustreError: dumping log to /tmp/lustre-log.1584493323.68579 [Tue Mar 17 18:02:03 2020][8592607.478545] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592607.484989] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:02:03 2020][8592607.490686] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592607.498271] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.503328] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.509617] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.516233] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.522507] kworker/10:2H S ffff9c6a29b08000 0 116594 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.529891] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.532524] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.538703] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.543859] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.549607] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592607.556307] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.561397] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 18:02:03 2020][8592607.567669] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.573942] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.580569] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.586839] kworker/45:2 S ffff9c6415de6180 0 80081 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.594111] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.596762] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.602941] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.608097] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.613845] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592607.620546] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.625634] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.631907] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.638520] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.644808] kworker/42:0H S ffff9c743d2a2080 0 122772 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.652169] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.654808] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.660991] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.666146] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.671894] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592607.678593] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.683660] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.689930] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.696542] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.702821] python S ffff9c786ad68000 0 128778 1 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.710181] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.712812] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.719520] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.724682] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 18:02:03 2020][8592607.731818] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:03 2020][8592607.737744] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 18:02:03 2020][8592607.745065] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:03 2020][8592607.751595] [] ep_poll+0x23e/0x360 [Tue Mar 17 18:02:03 2020][8592607.756828] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:03 2020][8592607.762590] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 18:02:03 2020][8592607.768338] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:03 2020][8592607.774524] kworker/0:1H S ffff9c50261a1040 0 92449 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.781817] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.784449] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.790629] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.795804] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.801558] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592607.808265] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.813333] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.819603] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.826223] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.832501] crond S ffff9c67616930c0 0 64012 50991 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.839775] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.842403] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.847546] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:02:03 2020][8592607.852808] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:03 2020][8592607.858818] [] pipe_read+0x298/0x400 [Tue Mar 17 18:02:03 2020][8592607.864217] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:02:03 2020][8592607.869718] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:02:03 2020][8592607.874949] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:02:03 2020][8592607.880096] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:03 2020][8592607.886298] raid-check S ffff9c83f055d140 0 64016 64012 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.893571] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.896200] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.901349] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:02:03 2020][8592607.906609] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:02:03 2020][8592607.911928] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:02:03 2020][8592607.918046] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:03 2020][8592607.924226] kworker/46:1H S ffff9c7a03b730c0 0 51157 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.931500] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.934134] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.940335] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592607.945477] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592607.951225] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592607.957946] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592607.963016] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.969285] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592607.975911] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592607.982182] kworker/18:1H S ffff9c7a09ed1040 0 103791 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592607.989548] Call Trace: [Tue Mar 17 18:02:03 2020][8592607.992182] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592607.998376] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.003519] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592608.009273] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592608.015995] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592608.021067] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.027334] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592608.033963] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.040231] ll_ost_out02_00 S ffff9c6d77688000 0 105890 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.047594] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.050223] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:03 2020][8592608.056335] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.061511] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.068388] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:03 2020][8592608.074178] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.080662] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.088235] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592608.093304] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.099572] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592608.106192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.112472] ll_ost_out03_00 S ffff9c7bb01e2080 0 107886 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.119833] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.122462] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:03 2020][8592608.128561] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.133745] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.140632] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:03 2020][8592608.146410] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.152898] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:03 2020][8592608.160466] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592608.165528] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.171815] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592608.178430] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.184711] kworker/38:2H S ffff9c6d6a60e180 0 10254 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.191987] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.194620] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:03 2020][8592608.200809] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.205985] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:03 2020][8592608.211736] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:03 2020][8592608.218436] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592608.223513] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.229781] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592608.236402] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.242683] crond S ffff9c609e29b0c0 0 55849 50991 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.249953] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.252584] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.257730] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:02:03 2020][8592608.262973] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:03 2020][8592608.268979] [] pipe_read+0x298/0x400 [Tue Mar 17 18:02:03 2020][8592608.274379] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:02:03 2020][8592608.279878] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:02:03 2020][8592608.285108] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:02:03 2020][8592608.290250] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:03 2020][8592608.296452] raid-check S ffff9c5392bfe180 0 55852 55849 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.303722] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.306353] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.311501] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:02:03 2020][8592608.316748] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:02:03 2020][8592608.322065] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:02:03 2020][8592608.328166] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:03 2020][8592608.334367] md0_resync D ffff9c5392bf8000 0 55899 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.341639] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.344269] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.349418] [] md_do_sync+0xddd/0x11d0 [Tue Mar 17 18:02:03 2020][8592608.355032] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:03 2020][8592608.361040] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:03 2020][8592608.366448] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:03 2020][8592608.371885] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:03 2020][8592608.376941] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.383221] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:03 2020][8592608.389840] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:03 2020][8592608.396113] md8_resync D ffff9c5d8f32b0c0 0 55908 2 0x00000080 [Tue Mar 17 18:02:03 2020][8592608.403398] Call Trace: [Tue Mar 17 18:02:03 2020][8592608.406025] [] schedule+0x29/0x70 [Tue Mar 17 18:02:03 2020][8592608.411167] [] md_do_sync+0xddd/0x11d0 [Tue Mar 17 18:02:03 2020][8592608.416763] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:03 2020][8592608.422786] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:03 2020][8592608.428190] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:03 2020][8592608.433609] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.438665] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.444947] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.451561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.457829] ll_ost_create03 S ffff9c4a3b96c100 0 10365 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.465122] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.467752] [] ? recalc_sigpending+0x1b/0x70 [Tue Mar 17 18:02:04 2020][8592608.473845] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.479065] [] ptlrpc_wait_event+0x345/0x360 [ptlrpc] [Tue Mar 17 18:02:04 2020][8592608.485945] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:04 2020][8592608.491743] [] ptlrpc_main+0xa02/0x1460 [ptlrpc] [Tue Mar 17 18:02:04 2020][8592608.492327] LustreError: dumping log to /tmp/lustre-log.1584493324.40806 [Tue Mar 17 18:02:04 2020][8592608.505080] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:02:04 2020][8592608.510790] [] ? ptlrpc_register_service+0xf80/0xf80 [ptlrpc] [Tue Mar 17 18:02:04 2020][8592608.518358] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.523434] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.529701] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.536319] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.542618] kworker/25:0 S ffff9c6a2cedb0c0 0 109877 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.549981] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.552607] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592608.558788] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.563950] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592608.569700] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592608.576420] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.581478] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.587752] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.594388] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.600671] kworker/20:1 S ffff9c5a39ea0000 0 98374 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.607946] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.610593] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592608.616775] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.621929] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592608.627679] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592608.634388] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.639461] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.645731] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.652345] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.658647] kworker/23:2 S ffff9c86ccb5e180 0 106607 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.666012] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.668638] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592608.674816] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.679970] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592608.685721] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592608.692430] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.697505] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.703774] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.710407] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.716691] kworker/5:1 S ffff9c6403f230c0 0 24319 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.723966] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.726605] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592608.732790] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.737951] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592608.743701] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592608.750409] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.755485] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.761755] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.768375] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.774669] crond S ffff9c4a66794100 0 24350 50991 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.781944] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.784573] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.789722] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:02:04 2020][8592608.794971] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:04 2020][8592608.800978] [] pipe_read+0x298/0x400 [Tue Mar 17 18:02:04 2020][8592608.806379] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:02:04 2020][8592608.811887] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:02:04 2020][8592608.817115] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:02:04 2020][8592608.822256] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:04 2020][8592608.828470] raid-check S ffff9c5baad65140 0 24351 24350 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.835739] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.838381] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.843524] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:02:04 2020][8592608.848765] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:02:04 2020][8592608.854081] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:02:04 2020][8592608.860181] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:04 2020][8592608.866375] md10_resync D ffff9c6403f24100 0 24401 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.873647] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.876276] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.881426] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:02:04 2020][8592608.888927] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:04 2020][8592608.894938] [] raid5_sync_request+0x417/0x430 [raid456] [Tue Mar 17 18:02:04 2020][8592608.901990] [] md_do_sync+0xb02/0x11d0 [Tue Mar 17 18:02:04 2020][8592608.907586] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:04 2020][8592608.913593] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:04 2020][8592608.918993] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:04 2020][8592608.924405] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.929463] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.935730] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592608.942380] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.948650] md6_resync D ffff9c50359b2080 0 24408 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592608.955919] Call Trace: [Tue Mar 17 18:02:04 2020][8592608.958561] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592608.963707] [] md_do_sync+0xddd/0x11d0 [Tue Mar 17 18:02:04 2020][8592608.969300] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:04 2020][8592608.975314] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:04 2020][8592608.980717] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:04 2020][8592608.986131] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592608.991188] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592608.997490] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.004109] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.010375] kworker/29:2 S ffff9c67ba341040 0 66555 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.017665] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.020290] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592609.026470] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.031632] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.037383] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.044090] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.049165] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.055447] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.062064] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.068345] kworker/30:2H S ffff9c7524968000 0 45745 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.075628] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.078261] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592609.084444] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.089603] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.095352] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.102067] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.107124] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.113397] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.120039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.126312] kworker/28:1 S ffff9c4a19a8b0c0 0 86933 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.133593] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.136226] [] ? start_worker+0x27/0x30 [Tue Mar 17 18:02:04 2020][8592609.141909] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.148611] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.153760] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.159526] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.166229] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.171282] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.177563] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.184177] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.190445] kworker/11:0 S ffff9c8a1eac6180 0 94215 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.197743] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.200375] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592609.206556] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.211718] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.217468] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.224174] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.229241] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.235510] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.242124] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.248414] kworker/1:0 S ffff9c6a1db88000 0 98441 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.255688] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.258324] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592609.264510] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.269664] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.275413] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.282121] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.287211] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.293485] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.300105] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.306393] kworker/27:2 S ffff9c89945e1040 0 11047 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.313668] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.316302] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.321442] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.327203] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.333904] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.338958] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.345260] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.351879] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.358174] kworker/3:2 S ffff9c870e4ec100 0 37556 2 0x00000080 [Tue Mar 17 18:02:04 2020][8592609.365469] Call Trace: [Tue Mar 17 18:02:04 2020][8592609.368094] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:04 2020][8592609.374292] [] schedule+0x29/0x70 [Tue Mar 17 18:02:04 2020][8592609.379455] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:04 2020][8592609.385205] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:04 2020][8592609.391925] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:04 2020][8592609.396981] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:04 2020][8592609.403256] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:04 2020][8592609.409903] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.416170] kworker/34:2H S ffff9c72a7cd2080 0 39994 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.423441] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.426088] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592609.432272] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.437426] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.443174] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.449873] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.454942] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.461226] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.467839] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.474120] kworker/43:1 S ffff9c8a2d8f6180 0 69254 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.481411] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.484037] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592609.490218] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.495380] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.501145] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.507847] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.512921] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.519204] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.525820] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.532101] kworker/31:0 S ffff9c8a35311040 0 96967 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.539377] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.542011] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.547158] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.552919] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.559619] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.564672] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.570954] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.577584] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.583853] kworker/21:1 S ffff9c6a37fe6180 0 102099 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.591229] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.593863] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592609.600043] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.605197] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.610946] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.617652] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.622718] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.628989] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.635601] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.641897] kworker/13:0 S ffff9c69b70730c0 0 110312 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.649261] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.651887] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592609.658075] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.663238] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.668987] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.675693] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.680762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.687030] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.693644] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.699932] crond S ffff9c5d8f32e180 0 122971 50991 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.707290] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.709918] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.715059] [] pipe_wait+0x70/0xc0 [Tue Mar 17 18:02:05 2020][8592609.720307] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:05 2020][8592609.726323] [] pipe_read+0x298/0x400 [Tue Mar 17 18:02:05 2020][8592609.731724] [] do_sync_read+0x93/0xe0 [Tue Mar 17 18:02:05 2020][8592609.737225] [] vfs_read+0x9f/0x170 [Tue Mar 17 18:02:05 2020][8592609.742456] [] SyS_read+0x7f/0xf0 [Tue Mar 17 18:02:05 2020][8592609.747603] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:05 2020][8592609.753803] raid-check S ffff9c67ba340000 0 122974 122971 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.761161] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.763792] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.768931] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:02:05 2020][8592609.774181] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:02:05 2020][8592609.779495] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:02:05 2020][8592609.785591] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:05 2020][8592609.791790] md2_resync D ffff9c62b0cf4100 0 123020 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.799148] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.801778] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.806928] [] raid5_get_active_stripe+0x4b9/0x6e0 [raid456] [Tue Mar 17 18:02:05 2020][8592609.814430] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:05 2020][8592609.820440] [] raid5_sync_request+0x417/0x430 [raid456] [Tue Mar 17 18:02:05 2020][8592609.827493] [] md_do_sync+0xb02/0x11d0 [Tue Mar 17 18:02:05 2020][8592609.833086] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:05 2020][8592609.839097] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:05 2020][8592609.844521] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:05 2020][8592609.849922] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.854974] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.861268] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.867887] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.874156] md4_resync D ffff9c62b0cf30c0 0 123029 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.881534] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.884166] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.889311] [] md_do_sync+0xddd/0x11d0 [Tue Mar 17 18:02:05 2020][8592609.894900] [] ? wake_up_atomic_t+0x30/0x30 [Tue Mar 17 18:02:05 2020][8592609.900907] [] md_thread+0x16d/0x1e0 [Tue Mar 17 18:02:05 2020][8592609.906308] [] ? find_pers+0x80/0x80 [Tue Mar 17 18:02:05 2020][8592609.911730] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.916800] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.923070] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.929704] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.935973] kworker/12:1 S ffff9c50261a2080 0 124266 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592609.943333] Call Trace: [Tue Mar 17 18:02:05 2020][8592609.945980] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592609.952163] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592609.957319] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592609.963066] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592609.969775] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592609.974862] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.981137] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592609.987771] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592609.994041] kworker/28:0 S ffff9c5a3a3dd140 0 9564 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.001313] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.003961] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.010143] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.015297] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.021047] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.027753] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.032834] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.039108] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.045722] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.052008] kworker/4:0 S ffff9c4a0b98c100 0 11825 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.059285] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.061920] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.068106] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.073261] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.079008] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.085710] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.090798] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.097070] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.103697] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.109967] kworker/43:2 S ffff9c801ddc6180 0 12370 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.117240] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.119886] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.125031] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.130800] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.137501] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.142561] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.148856] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.155476] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.161750] kworker/26:2H S ffff9c7a38ace180 0 18520 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.169040] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.171675] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.177862] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.183024] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.188775] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.195482] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.200557] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.206840] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.213457] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.219743] kworker/44:2 S ffff9c58b5999040 0 29036 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.227022] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.229671] [] ? ib_cq_completion_workqueue+0x1b/0x20 [ib_core] [Tue Mar 17 18:02:05 2020][8592610.237411] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.243604] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.248748] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.254501] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.261215] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.266286] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.272554] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.279181] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.285450] kworker/44:0 S ffff9c5a162c1040 0 31770 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.292723] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.295373] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.301554] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.306716] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.312465] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.319173] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.324263] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.330534] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.337147] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.343427] kworker/8:1 S ffff9c4f6e17c100 0 38456 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.350703] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.353329] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.359518] [] schedule+0x29/0x70 [Tue Mar 17 18:02:05 2020][8592610.364679] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:05 2020][8592610.370427] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:05 2020][8592610.377127] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:05 2020][8592610.382208] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.388480] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:05 2020][8592610.395095] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:05 2020][8592610.401382] kworker/47:0 S ffff9c7ae98b2080 0 45049 2 0x00000080 [Tue Mar 17 18:02:05 2020][8592610.408658] Call Trace: [Tue Mar 17 18:02:05 2020][8592610.411292] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:05 2020][8592610.417493] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.422636] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.428383] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.435105] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.440175] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.446443] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.453070] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.459340] kworker/40:2 S ffff9c4a1b739040 0 48501 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.466612] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.469246] [] ? start_worker+0x27/0x30 [Tue Mar 17 18:02:06 2020][8592610.474920] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.481623] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.486786] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.492547] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.499250] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.504348] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.510620] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.517234] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.523515] kworker/47:1 S ffff9c7bb01e5140 0 56815 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.530790] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.533422] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.538562] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.540339] LustreError: dumping log to /tmp/lustre-log.1584493326.68344 [Tue Mar 17 18:02:06 2020][8592610.551221] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.557938] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.562995] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.569283] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.575898] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.582166] kworker/29:0 S ffff9c66f1e99040 0 63549 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.589464] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.592097] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.597236] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.603004] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.609709] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.614769] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.621065] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.627684] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.633971] kworker/24:0 S ffff9c4b6976b0c0 0 64903 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.641248] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.643881] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.650070] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.655233] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.660981] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.667688] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.672767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.679033] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.685646] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.691925] kworker/7:0 S ffff9c870e4eb0c0 0 73766 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.699200] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.701827] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.708013] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.713169] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.718918] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.725624] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.730707] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.736979] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.743592] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.749879] kworker/19:1 S ffff9c8a3ecaa080 0 75494 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.757155] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.759791] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.765977] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.771132] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.776880] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.783581] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.788668] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.794940] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.801568] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.807839] kworker/15:2 S ffff9c8a1eac1040 0 77964 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.815120] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.817766] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.823950] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.829111] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.834862] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.841568] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.846649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.852922] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.859544] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.865833] kworker/4:1 S ffff9c4a2bc16180 0 90682 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.873109] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.875743] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.880890] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.886660] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.893363] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.898421] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.904717] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.911336] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.917602] kworker/11:1 S ffff9c801ddc4100 0 103067 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.924978] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.927613] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.933801] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.938962] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592610.944712] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592610.951421] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592610.956496] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.962779] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592610.969393] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592610.975675] kworker/30:0 S ffff9c6ae98eb0c0 0 105246 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592610.983037] Call Trace: [Tue Mar 17 18:02:06 2020][8592610.985670] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592610.991849] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592610.997004] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.002754] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.009460] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.014527] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.020813] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.027426] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.033707] kworker/25:2 S ffff9c68eef26180 0 115897 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.041070] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.043704] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.048853] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.054620] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.061325] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.066384] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.072674] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.079288] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.085552] kworker/40:1 S ffff9c4a2b57b0c0 0 118397 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.092931] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.095564] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592611.101743] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.106898] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.112649] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.119355] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.124423] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.130692] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.137312] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.143615] kworker/19:0 S ffff9c870e4ee180 0 124011 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.150980] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.153606] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.158753] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.164515] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.171218] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.176278] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.182556] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.189173] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.195467] kworker/14:2 S ffff9c7a31c98000 0 125105 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.202833] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.205459] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592611.211647] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.216810] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.222558] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.229265] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.234340] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.240612] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.247232] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.253533] kworker/41:1 S ffff9c6a362dd140 0 128506 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.260892] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.263526] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592611.269713] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.274869] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.280617] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.287326] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.292400] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.298671] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.305292] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.311593] kworker/35:2 S ffff9c7ae99f6180 0 130919 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.318959] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.321594] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592611.327780] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.332935] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.338685] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.345390] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.350459] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.356733] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:06 2020][8592611.363350] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.369640] kworker/2:2 S ffff9c74e7e69040 0 627 2 0x00000080 [Tue Mar 17 18:02:06 2020][8592611.376915] Call Trace: [Tue Mar 17 18:02:06 2020][8592611.379561] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:06 2020][8592611.385745] [] schedule+0x29/0x70 [Tue Mar 17 18:02:06 2020][8592611.390907] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:06 2020][8592611.396657] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:06 2020][8592611.403364] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:06 2020][8592611.408438] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:06 2020][8592611.414710] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.421330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.427628] kworker/23:1 S ffff9c7d5df76180 0 1238 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.434903] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.437550] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.442694] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.448463] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.455166] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.460226] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.466506] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.473121] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.479387] kworker/0:1 S ffff9c5a11398000 0 1566 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.486700] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.489328] [] ? start_worker+0x27/0x30 [Tue Mar 17 18:02:07 2020][8592611.494988] [] ? manage_workers.isra.25+0x189/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.501701] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.506844] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.512589] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.519305] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.524358] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.530625] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.537253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.543538] kworker/45:0 S ffff9c62b0cf5140 0 2020 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.550810] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.553451] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.558594] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.564361] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.571064] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.576132] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.582401] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.589012] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.595295] kworker/14:0 S ffff9c71fab9d140 0 2253 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.602587] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.605222] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.611409] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.616570] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.622318] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.629020] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.634094] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.640366] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.646986] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.653274] kworker/22:2 S ffff9c7a36e38000 0 3537 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.660568] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.663202] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.669388] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.674543] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.680293] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.687000] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.692074] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.698343] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.704970] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.711239] kworker/10:1 S ffff9c738578d140 0 3894 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.718512] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.721137] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.727350] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.732492] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.738244] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.744959] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.750015] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.756289] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.762918] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.769187] kworker/26:0 S ffff9c7a30dc9040 0 4122 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.776467] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.779100] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.785308] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.790453] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.796201] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.802922] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.807979] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.814253] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.820891] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.827157] kworker/3:1 S ffff9c7ae99f5140 0 4448 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.834430] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.837094] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.842237] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.847998] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.854701] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.859754] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.866043] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.872659] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.878933] kworker/33:2 S ffff9c6a39c49040 0 4521 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.886222] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.888847] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.895028] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.900203] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.905957] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.912665] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.917739] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.924010] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.930629] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.936911] kworker/9:2 S ffff9c6a39c4a080 0 4522 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592611.944184] Call Trace: [Tue Mar 17 18:02:07 2020][8592611.946810] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592611.952990] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592611.958143] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592611.963895] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592611.970617] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592611.975672] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.981959] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592611.988577] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592611.994849] kworker/17:2 S ffff9c6a39cda080 0 4523 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.002140] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.004772] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.010954] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.016114] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.021865] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.028573] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.033649] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.039916] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.046529] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.052810] kworker/41:2 S ffff9c609e29e180 0 4524 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.060086] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.062719] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.068899] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.074062] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.076347] LustreError: dumping log to /tmp/lustre-log.1584493327.68633 [Tue Mar 17 18:02:07 2020][8592612.086701] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.093422] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.098479] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.104768] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.111384] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.117659] kworker/32:0 S ffff9c5a11399040 0 4626 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.124964] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.127591] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.133777] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.138932] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.144679] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.151379] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.156447] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.162716] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.169339] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.175625] kworker/8:2 S ffff9c5a1139e180 0 4643 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.182920] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.185554] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.191742] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.196905] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.202654] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.209361] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.214438] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.220706] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.227329] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.233613] kworker/24:1 S ffff9c4a2b57e180 0 4645 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.240909] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.243544] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.249728] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.254882] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.260634] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.267340] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.272409] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.278677] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.285298] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.291586] kworker/34:2 S ffff9c6b54569040 0 4655 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.298880] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.301513] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.307693] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.312855] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.318604] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.325304] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.330373] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.336640] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.343253] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.349541] kworker/42:2 S ffff9c79c98ec100 0 4658 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.356817] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.359498] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:07 2020][8592612.365683] [] schedule+0x29/0x70 [Tue Mar 17 18:02:07 2020][8592612.370843] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:07 2020][8592612.376594] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:07 2020][8592612.383294] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:07 2020][8592612.388367] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.394638] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:07 2020][8592612.401259] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:07 2020][8592612.407546] kworker/18:2 S ffff9c6ae99d0000 0 4659 2 0x00000080 [Tue Mar 17 18:02:07 2020][8592612.414840] Call Trace: [Tue Mar 17 18:02:07 2020][8592612.417465] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.423646] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.428809] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.434557] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.441270] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.446330] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.452601] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.459228] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.465496] kworker/12:2 S ffff9c4a2b578000 0 4680 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.472770] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.475432] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.481618] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.486794] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.492547] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.499267] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.504324] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.510614] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.517242] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.523513] kworker/u257:2 S ffff9c5991101040 0 4710 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.530820] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.533454] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.539640] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.544795] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.550545] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.557252] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.562344] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.568613] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.575226] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.581508] kworker/39:0 S ffff9c8a351830c0 0 4805 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.588341] LustreError: dumping log to /tmp/lustre-log.1584493328.68594 [Tue Mar 17 18:02:08 2020][8592612.595673] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.598307] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.604495] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.609657] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.615406] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.622126] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.627183] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.633449] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.640077] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.646361] kworker/33:1 S ffff9c6459194100 0 5248 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.653635] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.656261] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.661424] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.667171] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.673873] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.678947] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.685216] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.691852] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.698121] kworker/22:1 S ffff9c74e7e6d140 0 6330 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.705402] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.708064] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.714250] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.719412] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.725161] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.731870] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.736946] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.743214] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.749835] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.756123] kworker/36:0 S ffff9c4a19414100 0 6468 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.763416] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.766042] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.772230] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.777384] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.783131] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.789831] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.794900] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.801168] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.807779] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.814059] kworker/20:0 S ffff9c4a19411040 0 7172 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.821353] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.823978] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.829119] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.834887] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.841590] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.846650] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.852931] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.859547] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.865837] kworker/6:2 R running task 0 7183 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.873111] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.875747] [] ? md_submit_flush_data+0x5a/0x70 [Tue Mar 17 18:02:08 2020][8592612.882107] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.888322] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.893467] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.899228] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.905930] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.910997] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.917268] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.923889] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.930169] kworker/39:1 S ffff9c7d5df730c0 0 8167 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.937462] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.940096] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592612.945236] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592612.951004] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592612.957706] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592612.962759] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.969047] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592612.975663] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592612.981928] kworker/30:1 S ffff9c6b5456a080 0 9194 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592612.989218] Call Trace: [Tue Mar 17 18:02:08 2020][8592612.991844] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592612.998024] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.003192] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.008945] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.015667] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.020723] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.026997] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.033632] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.039901] kworker/46:1 S ffff9c6b5456e180 0 9243 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.047172] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.049814] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592613.055995] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.061149] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.066900] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.073605] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.078673] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.084941] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.091556] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.097844] kworker/17:0 S ffff9c66f1e98000 0 9305 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.105120] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.107754] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.112902] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.118661] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.125366] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.130425] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.136704] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.143322] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.149596] kworker/0:2 S ffff9c5a28628000 0 9666 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.156887] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.159519] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:08 2020][8592613.165698] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.170853] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.176602] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.183310] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.188376] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.194646] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.201274] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.207544] kworker/32:2 S ffff9c5a16e1c100 0 9705 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.214823] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.217471] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.222612] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.228375] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.235076] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.240130] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.246418] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.253034] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.259322] kworker/6:0 S ffff9c79ca36a080 0 9754 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.266599] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.269232] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.274380] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.280152] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.286861] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.291923] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.298213] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.304826] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.311102] kworker/26:2 S ffff9c7a31c99040 0 10102 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.318391] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.321025] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.326164] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.331934] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.338650] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.343706] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.349995] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.356610] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.362896] kworker/2:0 S ffff9c7a36e3e180 0 10361 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.370191] Call Trace: [Tue Mar 17 18:02:08 2020][8592613.372827] [] schedule+0x29/0x70 [Tue Mar 17 18:02:08 2020][8592613.377974] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:08 2020][8592613.383741] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:08 2020][8592613.390446] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:08 2020][8592613.395507] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.401808] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:08 2020][8592613.408427] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:08 2020][8592613.414707] kworker/37:1 S ffff9c6a29b0a080 0 10852 2 0x00000080 [Tue Mar 17 18:02:08 2020][8592613.421984] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.424618] [] ? process_one_work+0x336/0x440 [Tue Mar 17 18:02:09 2020][8592613.430805] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.435958] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.441708] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.448408] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.453497] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.459769] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.466383] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.472670] kworker/21:0 S ffff9c66f1e9c100 0 11080 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.479947] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.482582] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.487730] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.493512] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.500218] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.505279] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.511572] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.518192] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.524474] kworker/18:1 S ffff9c6ec59fe180 0 11428 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.531766] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.534398] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.539538] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.545300] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.552002] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.557076] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.563345] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.569959] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.576261] kworker/u256:1 S ffff9c62b0cf6180 0 11485 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.583541] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.586176] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592613.592361] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.597516] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.603266] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.609973] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.612346] LustreError: dumping log to /tmp/lustre-log.1584493329.68548 [Tue Mar 17 18:02:09 2020][8592613.621916] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.628210] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.634828] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.641110] kworker/u257:0 S ffff9c5a182ed140 0 11538 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.648383] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.651011] [] ? __call_usermodehelper+0x4e/0x90 [Tue Mar 17 18:02:09 2020][8592613.657459] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592613.663660] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.668802] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.674563] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.681268] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.686328] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.692628] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.699239] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.705507] sshd S ffff9c77e8266180 0 11736 50881 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.712801] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.715431] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.720579] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 18:02:09 2020][8592613.727822] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 18:02:09 2020][8592613.733659] [] ? tty_write_room+0x20/0x30 [Tue Mar 17 18:02:09 2020][8592613.739498] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:09 2020][8592613.746061] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 18:02:09 2020][8592613.752338] [] do_select+0x6d1/0x7c0 [Tue Mar 17 18:02:09 2020][8592613.757744] [] ? ip_finish_output+0x284/0x7b0 [Tue Mar 17 18:02:09 2020][8592613.763945] [] ? dequeue_entity+0x11c/0x5e0 [Tue Mar 17 18:02:09 2020][8592613.769953] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:09 2020][8592613.775448] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:09 2020][8592613.780955] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:09 2020][8592613.786446] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:09 2020][8592613.791940] [] ? _raw_spin_unlock_bh+0x1e/0x20 [Tue Mar 17 18:02:09 2020][8592613.798230] [] ? release_sock+0x120/0x170 [Tue Mar 17 18:02:09 2020][8592613.804080] [] ? tcp_sendmsg+0xe0/0xc60 [Tue Mar 17 18:02:09 2020][8592613.809747] [] ? tracing_record_cmdline+0x1d/0x120 [Tue Mar 17 18:02:09 2020][8592613.816378] [] ? inet_sendmsg+0x69/0xb0 [Tue Mar 17 18:02:09 2020][8592613.822039] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 18:02:09 2020][8592613.827984] [] ? do_sync_write+0x93/0xe0 [Tue Mar 17 18:02:09 2020][8592613.833730] [] SyS_select+0xca/0x120 [Tue Mar 17 18:02:09 2020][8592613.839129] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:09 2020][8592613.845324] bash S ffff9c77e82630c0 0 11738 11736 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.852596] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.855225] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.860373] [] do_wait+0x1f6/0x260 [Tue Mar 17 18:02:09 2020][8592613.865634] [] SyS_wait4+0x80/0x110 [Tue Mar 17 18:02:09 2020][8592613.870953] [] ? task_stopped_code+0x60/0x60 [Tue Mar 17 18:02:09 2020][8592613.877048] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:09 2020][8592613.883249] lnetctl R running task 0 11793 11738 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.890522] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.893154] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 18:02:09 2020][8592613.899685] [] ? sysret_audit+0x17/0x21 [Tue Mar 17 18:02:09 2020][8592613.905366] kworker/35:0 S ffff9c8a327d0000 0 11937 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.912642] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.915277] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592613.921480] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.926634] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.932385] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.939108] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592613.944161] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.950428] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592613.957056] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592613.963324] kworker/38:0 S ffff9c7312866180 0 11947 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592613.970597] Call Trace: [Tue Mar 17 18:02:09 2020][8592613.973230] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592613.979440] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592613.984582] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592613.990333] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592613.997051] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.002108] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.008373] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.015001] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.021269] kworker/37:2 S ffff9c6a223a30c0 0 11979 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.028541] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.031168] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592614.037369] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.042512] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.048267] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.054990] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.060045] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.066320] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.072954] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.079224] kworker/13:1 S ffff9c5d8f329040 0 12164 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.086497] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.089131] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.094292] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.100042] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.106741] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.111809] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.118077] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.124691] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.130981] kworker/5:2 S ffff9c5d8f32d140 0 12197 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.138254] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.140889] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592614.147077] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.152238] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.157988] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.164695] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.169762] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.176034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.182652] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.188932] kworker/u259:1 S ffff9c7a32889040 0 12614 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.196208] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.198847] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592614.205030] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.210184] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.215933] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.222632] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.227700] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.233968] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.240581] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.246860] kworker/u258:0 S ffff9c66f1e9a080 0 12753 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.254137] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.256774] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:02:09 2020][8592614.263392] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592614.269586] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.274728] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.280476] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.287198] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.292254] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.298528] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.305165] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.311432] kworker/42:1 S ffff9c7a27d45140 0 12971 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.318722] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.321356] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.326505] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.332286] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.338993] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.344069] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.350338] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.356959] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.363266] kworker/16:2 S ffff9c4a0389d140 0 13062 2 0x00000080 [Tue Mar 17 18:02:09 2020][8592614.370541] Call Trace: [Tue Mar 17 18:02:09 2020][8592614.373173] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:09 2020][8592614.379354] [] schedule+0x29/0x70 [Tue Mar 17 18:02:09 2020][8592614.384516] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:09 2020][8592614.390266] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:09 2020][8592614.396971] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:09 2020][8592614.402039] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.408309] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:09 2020][8592614.414922] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:09 2020][8592614.421211] kworker/u258:3 S ffff9c68eef230c0 0 13072 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.428486] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.431122] [] ? queued_spin_lock_slowpath+0xb/0xf [Tue Mar 17 18:02:10 2020][8592614.437743] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.443943] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.449087] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.454856] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.461557] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.466619] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.472900] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.479517] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.485790] pickup S ffff9c581aff2080 0 13120 51202 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.493097] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.495732] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.502438] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.507615] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 18:02:10 2020][8592614.514755] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:10 2020][8592614.520683] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 18:02:10 2020][8592614.528010] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:10 2020][8592614.534544] [] ep_poll+0x23e/0x360 [Tue Mar 17 18:02:10 2020][8592614.539776] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:10 2020][8592614.545543] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 18:02:10 2020][8592614.551293] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:10 2020][8592614.557481] kworker/7:2 S ffff9c89945e0000 0 13142 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.564778] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.567405] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.573592] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.578743] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.584494] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.591192] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.596263] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.602532] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.609153] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.615453] kworker/15:0 S ffff9c89945e4100 0 13144 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.622735] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.625368] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.631556] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.636342] LustreError: dumping log to /tmp/lustre-log.1584493330.68476 [Tue Mar 17 18:02:10 2020][8592614.643593] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.649368] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.656072] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.661140] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.667411] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.674031] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.680318] kworker/u259:2 S ffff9c7a09ed4100 0 13272 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.687595] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.690228] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.696410] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.701570] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.707319] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.714033] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.719090] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.725365] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.732012] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.738284] kworker/u258:2 S ffff9c69c05ae180 0 13341 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.745559] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.748192] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.754393] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.759537] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.765282] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.771996] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.777051] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.783318] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.789966] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.796233] kworker/u259:3 S ffff9c7a31c9b0c0 0 13342 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.803511] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.806137] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.812337] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.817482] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.823230] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.829952] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.835005] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.841272] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.847914] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.854184] kworker/1:1 S ffff9c5ae9676180 0 13366 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.861457] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.864099] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.869240] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.875001] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.881701] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.886755] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.893036] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.899652] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.905928] kworker/u260:0 S ffff9c7bb01e30c0 0 13411 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.913232] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.915858] [] ? __call_usermodehelper+0x4e/0x90 [Tue Mar 17 18:02:10 2020][8592614.922298] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592614.928499] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.933644] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.939398] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.946120] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592614.951175] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.957444] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592614.964088] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592614.970354] kworker/46:2 S ffff9c7a36e39040 0 13418 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592614.977644] Call Trace: [Tue Mar 17 18:02:10 2020][8592614.980278] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592614.985427] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592614.991196] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592614.997897] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.002949] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.009230] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.015845] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.022111] kworker/36:2 S ffff9c5a277d0000 0 13435 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.029411] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.032044] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592615.038225] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.043385] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.049138] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.055845] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.060920] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.067191] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.073810] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.080112] kworker/34:0 S ffff9c6b5456d140 0 13494 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.087391] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.090026] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592615.096212] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.101367] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.107115] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.113816] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.118891] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.125161] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.131782] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.138084] kworker/u256:2 S ffff9c6b5456c100 0 13499 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.145362] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.147988] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592615.154176] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.159331] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.165087] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.171801] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.176856] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.183137] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.189753] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.196027] kworker/u258:4 S ffff9c5ae96730c0 0 13546 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.203326] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.205960] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592615.212148] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.217310] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.223061] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.229766] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.234835] [] ? finish_task_switch+0x54/0x1c0 [Tue Mar 17 18:02:10 2020][8592615.241102] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.247371] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.254018] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.260292] kworker/10:2 S ffff9c7a27d46180 0 13676 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.267579] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.270208] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.275354] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.281116] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.287815] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.292871] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.299158] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.305776] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.312075] kworker/u262:2 S ffff9c5a277d1040 0 13690 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.319346] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.321973] [] ? process_one_work+0x336/0x440 [Tue Mar 17 18:02:10 2020][8592615.328161] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.333321] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.339072] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.345781] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.350857] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.357124] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.363753] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.370033] kworker/16:0 S ffff9c50261a6180 0 13721 2 0x00000080 [Tue Mar 17 18:02:10 2020][8592615.377308] Call Trace: [Tue Mar 17 18:02:10 2020][8592615.379950] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:10 2020][8592615.386132] [] schedule+0x29/0x70 [Tue Mar 17 18:02:10 2020][8592615.391285] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:10 2020][8592615.397037] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:10 2020][8592615.403743] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:10 2020][8592615.408809] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:10 2020][8592615.415078] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:10 2020][8592615.421693] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.427992] kworker/9:1 S ffff9c5ae9675140 0 13754 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.435264] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.437900] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.443047] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.448815] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.455520] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.460579] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.466860] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.473477] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.479751] kworker/u259:0 S ffff9c7a27d41040 0 13851 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.487066] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.489700] [] ? __call_usermodehelper+0x4e/0x90 [Tue Mar 17 18:02:11 2020][8592615.496162] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.502357] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.507502] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.513271] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.519973] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.525035] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.531325] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.537939] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.544213] kworker/u262:0 S ffff9c5a277d6180 0 13913 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.551519] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.554162] [] ? ib_cq_poll_work+0x45/0x60 [ib_core] [Tue Mar 17 18:02:11 2020][8592615.560949] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.567144] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.572285] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.578031] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.584744] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.589801] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.596076] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.602726] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.608998] kworker/u256:3 S ffff9c5a2bf78000 0 13933 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.616278] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.618925] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.625110] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.630271] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.636020] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.642729] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.647805] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.654074] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.660695] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.666997] kworker/u260:1 S ffff9c7bb01e4100 0 13948 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.674276] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.676909] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.682048] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.687811] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.694511] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.699564] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.705847] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.712461] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.718725] kworker/u258:1 S ffff9c6a36d15140 0 13961 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.726016] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.728651] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.734840] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.740000] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.745750] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.752471] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.757529] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.763802] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.770430] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.776698] kworker/u259:4 S ffff9c786ad69040 0 13962 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.783971] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.786618] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.792803] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.797963] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.803711] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.810413] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.815487] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.821755] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.828368] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.834664] kworker/38:1 S ffff9c786ad6e180 0 13980 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.841943] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.844576] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.850754] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.855909] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.861658] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.868359] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.873433] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.879704] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.886341] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.892630] kworker/u258:5 S ffff9c69b7070000 0 14038 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.899906] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.902553] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:11 2020][8592615.908736] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.913889] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592615.919639] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592615.926347] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592615.931415] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.937683] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592615.944305] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592615.950606] sshd S ffff9c4a66796180 0 14039 50881 0x00000080 [Tue Mar 17 18:02:11 2020][8592615.957881] Call Trace: [Tue Mar 17 18:02:11 2020][8592615.960512] [] ? update_curr+0x14c/0x1e0 [Tue Mar 17 18:02:11 2020][8592615.966268] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592615.971429] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 18:02:11 2020][8592615.978651] [] ? add_wait_queue+0x3c/0x50 [Tue Mar 17 18:02:11 2020][8592615.984494] [] ? tty_write_room+0x20/0x30 [Tue Mar 17 18:02:11 2020][8592615.990350] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:11 2020][8592615.996880] [] poll_schedule_timeout+0x55/0xb0 [Tue Mar 17 18:02:11 2020][8592616.003152] [] do_select+0x6d1/0x7c0 [Tue Mar 17 18:02:11 2020][8592616.008589] [] ? ip_finish_output+0x284/0x7b0 [Tue Mar 17 18:02:11 2020][8592616.014775] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:11 2020][8592616.020269] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:11 2020][8592616.025779] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:11 2020][8592616.031268] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:11 2020][8592616.036762] [] ? tracing_record_cmdline+0x1d/0x120 [Tue Mar 17 18:02:11 2020][8592616.043398] [] ? probe_sched_wakeup+0x2b/0xa0 [Tue Mar 17 18:02:11 2020][8592616.049581] [] ? ttwu_do_wakeup+0xb5/0xe0 [Tue Mar 17 18:02:11 2020][8592616.055422] [] ? tracing_is_on+0x15/0x30 [Tue Mar 17 18:02:11 2020][8592616.061188] [] ? tracing_record_cmdline+0x1d/0x120 [Tue Mar 17 18:02:11 2020][8592616.067819] [] ? probe_sched_wakeup+0x2b/0xa0 [Tue Mar 17 18:02:11 2020][8592616.074000] [] ? ttwu_do_wakeup+0xb5/0xe0 [Tue Mar 17 18:02:11 2020][8592616.079849] [] ? try_to_wake_up+0x190/0x390 [Tue Mar 17 18:02:11 2020][8592616.085860] [] ? default_wake_function+0x12/0x20 [Tue Mar 17 18:02:11 2020][8592616.092307] [] ? __wake_up_common+0x5b/0x90 [Tue Mar 17 18:02:11 2020][8592616.098342] [] core_sys_select+0x1ff/0x340 [Tue Mar 17 18:02:11 2020][8592616.104266] [] ? __wake_up+0x44/0x50 [Tue Mar 17 18:02:11 2020][8592616.109673] [] ? tty_write_unlock+0x31/0x40 [Tue Mar 17 18:02:11 2020][8592616.115696] [] ? tty_ldisc_deref+0x19/0x20 [Tue Mar 17 18:02:11 2020][8592616.121617] [] ? tty_write+0x1eb/0x2c0 [Tue Mar 17 18:02:11 2020][8592616.127191] [] ? n_tty_ioctl+0x100/0x100 [Tue Mar 17 18:02:11 2020][8592616.132972] [] SyS_select+0xca/0x120 [Tue Mar 17 18:02:11 2020][8592616.138377] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:11 2020][8592616.144571] bash R running task 0 14041 14039 0x00000088 [Tue Mar 17 18:02:11 2020][8592616.151843] Call Trace: [Tue Mar 17 18:02:11 2020][8592616.154472] [] sched_show_task+0xa8/0x110 [Tue Mar 17 18:02:11 2020][8592616.160305] [] show_state_filter+0x82/0xd0 [Tue Mar 17 18:02:11 2020][8592616.166240] [] sysrq_handle_showstate+0x10/0x20 [Tue Mar 17 18:02:11 2020][8592616.172594] [] __handle_sysrq+0x10d/0x170 [Tue Mar 17 18:02:11 2020][8592616.178426] [] write_sysrq_trigger+0x28/0x40 [Tue Mar 17 18:02:11 2020][8592616.184549] [] proc_reg_write+0x40/0x80 [Tue Mar 17 18:02:11 2020][8592616.190215] [] vfs_write+0xc0/0x1f0 [Tue Mar 17 18:02:11 2020][8592616.195535] [] SyS_write+0x7f/0xf0 [Tue Mar 17 18:02:11 2020][8592616.200776] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:11 2020][8592616.206957] kworker/u262:1 S ffff9c5991100000 0 14168 2 0x00000080 [Tue Mar 17 18:02:11 2020][8592616.214231] Call Trace: [Tue Mar 17 18:02:11 2020][8592616.216876] [] ? process_one_work+0x336/0x440 [Tue Mar 17 18:02:11 2020][8592616.223062] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592616.228223] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:11 2020][8592616.233973] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:11 2020][8592616.240682] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:11 2020][8592616.245767] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592616.252034] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:11 2020][8592616.258647] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:11 2020][8592616.264927] sleep S ffff9c6a01fea080 0 14234 122974 0x00000080 [Tue Mar 17 18:02:11 2020][8592616.272198] Call Trace: [Tue Mar 17 18:02:11 2020][8592616.274842] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:11 2020][8592616.281544] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592616.286698] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 18:02:11 2020][8592616.292187] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 18:02:11 2020][8592616.298203] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:11 2020][8592616.304145] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 18:02:11 2020][8592616.309805] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 18:02:11 2020][8592616.315379] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:11 2020][8592616.321583] sleep S ffff9c7ae98e0000 0 14254 55852 0x00000080 [Tue Mar 17 18:02:11 2020][8592616.328852] Call Trace: [Tue Mar 17 18:02:11 2020][8592616.331483] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:11 2020][8592616.338190] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592616.343348] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 18:02:11 2020][8592616.348842] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 18:02:11 2020][8592616.354858] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:11 2020][8592616.360800] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 18:02:11 2020][8592616.366462] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 18:02:11 2020][8592616.372042] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:11 2020][8592616.378236] sleep S ffff9c4ddbdf1040 0 14266 64016 0x00000080 [Tue Mar 17 18:02:11 2020][8592616.385507] Call Trace: [Tue Mar 17 18:02:11 2020][8592616.388138] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:11 2020][8592616.394845] [] schedule+0x29/0x70 [Tue Mar 17 18:02:11 2020][8592616.400008] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 18:02:11 2020][8592616.405496] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 18:02:11 2020][8592616.411502] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:11 2020][8592616.417458] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 18:02:11 2020][8592616.423124] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 18:02:12 2020][8592616.428695] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.434890] cleanup S ffff9c68d2bc1040 0 14288 51202 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.442162] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.444792] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.451500] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.456662] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 18:02:12 2020][8592616.463799] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:12 2020][8592616.469725] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 18:02:12 2020][8592616.477045] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:12 2020][8592616.483575] [] ep_poll+0x23e/0x360 [Tue Mar 17 18:02:12 2020][8592616.488808] [] ? hrtimer_start+0x12/0x20 [Tue Mar 17 18:02:12 2020][8592616.494570] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:12 2020][8592616.500333] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 18:02:12 2020][8592616.506089] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.512285] trivial-rewrite S ffff9c68d2bc2080 0 14290 51202 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.519573] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.522201] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.528916] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.534059] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 18:02:12 2020][8592616.541200] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:12 2020][8592616.547140] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 18:02:12 2020][8592616.554451] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:12 2020][8592616.560978] [] ep_poll+0x23e/0x360 [Tue Mar 17 18:02:12 2020][8592616.566225] [] ? hrtimer_start+0x12/0x20 [Tue Mar 17 18:02:12 2020][8592616.571975] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:12 2020][8592616.577729] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 18:02:12 2020][8592616.583490] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.589687] smtp S ffff9c68d2bc30c0 0 14291 51202 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.596959] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.599601] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.606304] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.611456] [] schedule_hrtimeout_range_clock+0xb2/0x150 [Tue Mar 17 18:02:12 2020][8592616.618592] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:12 2020][8592616.624513] [] ? schedule_hrtimeout_range_clock+0xa6/0x150 [Tue Mar 17 18:02:12 2020][8592616.631842] [] schedule_hrtimeout_range+0x13/0x20 [Tue Mar 17 18:02:12 2020][8592616.638371] [] ep_poll+0x23e/0x360 [Tue Mar 17 18:02:12 2020][8592616.643606] [] ? wake_up_state+0x20/0x20 [Tue Mar 17 18:02:12 2020][8592616.649388] [] SyS_epoll_wait+0xed/0x120 [Tue Mar 17 18:02:12 2020][8592616.655141] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.661328] kworker/u256:0 S ffff9c58605dd140 0 14295 2 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.668619] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.671251] [] ? process_one_work+0x21a/0x440 [Tue Mar 17 18:02:12 2020][8592616.677431] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.682585] [] worker_thread+0x1d9/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.684356] LustreError: dumping log to /tmp/lustre-log.1584493332.70319 [Tue Mar 17 18:02:12 2020][8592616.695231] [] ? manage_workers.isra.25+0x2a0/0x2a0 [Tue Mar 17 18:02:12 2020][8592616.701945] [] kthread+0xd1/0xe0 [Tue Mar 17 18:02:12 2020][8592616.707002] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:12 2020][8592616.713281] [] ret_from_fork_nospec_begin+0xe/0x21 [Tue Mar 17 18:02:12 2020][8592616.719899] [] ? insert_kthread_work+0x40/0x40 [Tue Mar 17 18:02:12 2020][8592616.726164] sleep S ffff9c6a01fe8000 0 14300 24351 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.733450] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.736080] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.742786] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.747940] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 18:02:12 2020][8592616.753431] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 18:02:12 2020][8592616.759444] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:12 2020][8592616.765393] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 18:02:12 2020][8592616.771059] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 18:02:12 2020][8592616.776641] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.782844] sleep S ffff9c7b579ee180 0 14312 57403 0x00000080 [Tue Mar 17 18:02:12 2020][8592616.790115] Call Trace: [Tue Mar 17 18:02:12 2020][8592616.792745] [] ? hrtimer_start_range_ns+0x1ed/0x3c0 [Tue Mar 17 18:02:12 2020][8592616.799450] [] schedule+0x29/0x70 [Tue Mar 17 18:02:12 2020][8592616.804604] [] do_nanosleep+0x86/0xf0 [Tue Mar 17 18:02:12 2020][8592616.810094] [] hrtimer_nanosleep+0xbb/0x180 [Tue Mar 17 18:02:12 2020][8592616.816108] [] ? hrtimer_get_res+0x50/0x50 [Tue Mar 17 18:02:12 2020][8592616.822054] [] ? do_nanosleep+0x72/0xf0 [Tue Mar 17 18:02:12 2020][8592616.827723] [] SyS_nanosleep+0x96/0xb0 [Tue Mar 17 18:02:12 2020][8592616.833304] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:12 2020][8592616.839508] Sched Debug Version: v0.11, 3.10.0-957.27.2.el7_lustre.pl2.x86_64 #1 [Tue Mar 17 18:02:12 2020][8592616.847081] ktime : 8592594043.173278 [Tue Mar 17 18:02:12 2020][8592616.853867] sched_clk : 8592616839.507643 [Tue Mar 17 18:02:12 2020][8592616.860670] cpu_clk : 8592616839.507663 [Tue Mar 17 18:02:12 2020][8592616.867458] jiffies : 12887261367 [Tue Mar 17 18:02:12 2020][8592616.873723] sched_clock_stable() : 1 [Tue Mar 17 18:02:12 2020][8592616.879137] [Tue Mar 17 18:02:12 2020][8592616.880812] sysctl_sched [Tue Mar 17 18:02:12 2020][8592616.883539] .sysctl_sched_latency : 24.000000 [Tue Mar 17 18:02:12 2020][8592616.889897] .sysctl_sched_min_granularity : 10.000000 [Tue Mar 17 18:02:12 2020][8592616.896272] .sysctl_sched_wakeup_granularity : 4.000000 [Tue Mar 17 18:02:12 2020][8592616.902541] .sysctl_sched_child_runs_first : 0 [Tue Mar 17 18:02:12 2020][8592616.908199] .sysctl_sched_features : 56955 [Tue Mar 17 18:02:12 2020][8592616.914221] .sysctl_sched_tunable_scaling : 1 (logaritmic) [Tue Mar 17 18:02:12 2020][8592616.921009] [Tue Mar 17 18:02:12 2020][8592616.922682] cpu#0, 1996.233 MHz [Tue Mar 17 18:02:12 2020][8592616.926001] .nr_running : 0 [Tue Mar 17 18:02:12 2020][8592616.930807] .load : 0 [Tue Mar 17 18:02:12 2020][8592616.935618] .nr_switches : 23367840710 [Tue Mar 17 18:02:12 2020][8592616.941280] .nr_load_updates : 3628660450 [Tue Mar 17 18:02:12 2020][8592616.946868] .nr_uninterruptible : -1880744 [Tue Mar 17 18:02:12 2020][8592616.952271] .next_balance : 12887.261453 [Tue Mar 17 18:02:12 2020][8592616.958027] .curr->pid : 0 [Tue Mar 17 18:02:12 2020][8592616.962840] .clock : 8592616962.336661 [Tue Mar 17 18:02:12 2020][8592616.969024] .cpu_load[0] : 0 [Tue Mar 17 18:02:12 2020][8592616.973824] .cpu_load[1] : 0 [Tue Mar 17 18:02:12 2020][8592616.978631] .cpu_load[2] : 0 [Tue Mar 17 18:02:12 2020][8592616.983441] .cpu_load[3] : 0 [Tue Mar 17 18:02:12 2020][8592616.988238] .cpu_load[4] : 0 [Tue Mar 17 18:02:12 2020][8592616.993038] .avg_idle : 1563276 [Tue Mar 17 18:02:12 2020][8592616.998366] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:12 2020][8592617.003680] [Tue Mar 17 18:02:12 2020][8592617.005351] cfs_rq[0]:/ [Tue Mar 17 18:02:12 2020][8592617.007975] .exec_clock : 0.000000 [Tue Mar 17 18:02:12 2020][8592617.013374] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:12 2020][8592617.018774] .min_vruntime : 317215134.501246 [Tue Mar 17 18:02:12 2020][8592617.024867] .max_vruntime : 0.000001 [Tue Mar 17 18:02:12 2020][8592617.030265] .spread : 0.000000 [Tue Mar 17 18:02:12 2020][8592617.035664] .spread0 : 0.000000 [Tue Mar 17 18:02:12 2020][8592617.041065] .nr_spread_over : 0 [Tue Mar 17 18:02:12 2020][8592617.045859] .nr_running : 0 [Tue Mar 17 18:02:12 2020][8592617.050650] .load : 0 [Tue Mar 17 18:02:12 2020][8592617.055442] .runnable_load_avg : 0 [Tue Mar 17 18:02:12 2020][8592617.060237] .blocked_load_avg : 3 [Tue Mar 17 18:02:12 2020][8592617.065028] .tg_load_avg : 0 [Tue Mar 17 18:02:12 2020][8592617.069821] .tg_load_contrib : 0 [Tue Mar 17 18:02:12 2020][8592617.074614] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:12 2020][8592617.079406] .tg->runnable_avg : 0 [Tue Mar 17 18:02:12 2020][8592617.084199] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:12 2020][8592617.088993] .throttled : 0 [Tue Mar 17 18:02:12 2020][8592617.093786] .throttle_count : 0 [Tue Mar 17 18:02:12 2020][8592617.098579] .avg->runnable_avg_sum : 126 [Tue Mar 17 18:02:12 2020][8592617.103545] .avg->runnable_avg_period : 46238 [Tue Mar 17 18:02:12 2020][8592617.108694] [Tue Mar 17 18:02:12 2020][8592617.110364] rt_rq[0]:/ [Tue Mar 17 18:02:12 2020][8592617.112904] .rt_nr_running : 0 [Tue Mar 17 18:02:12 2020][8592617.117697] .rt_throttled : 0 [Tue Mar 17 18:02:12 2020][8592617.122489] .rt_time : 0.000000 [Tue Mar 17 18:02:12 2020][8592617.127887] .rt_runtime : 910.829286 [Tue Mar 17 18:02:12 2020][8592617.133462] [Tue Mar 17 18:02:12 2020][8592617.135132] runnable tasks: [Tue Mar 17 18:02:12 2020][8592617.138105] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:12 2020][8592617.149051] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:12 2020][8592617.159914] ksoftirqd/0 3 317215109.791298 617265834 120 0.000000 12756656.120920 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.171247] kworker/0:0H 5 223355963.649026 13 100 0.000000 0.078177 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.182582] migration/0 8 0.000000 112285 0 0.000000 93814.956489 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.193919] rcu_bh 9 513.292678 2 120 0.000000 0.001523 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.205257] lru-add-drain 11 517.294233 2 100 0.000000 0.002053 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.216593] watchdog/0 12 0.000000 2148155 0 0.000000 24028.022458 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.227968] irq/26-AMD-Vi 268 0.000000 3 49 0.000000 0.040726 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.239306] mlx5_cmd_0000:0 2657 3516.087785 2 100 0.000000 0.010219 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.250690] kdmwork-253:18 12671 317215016.901820 873470320 120 0.000000 30296221.297504 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.262030] kdmwork-253:31 12712 905136.475338 95574 120 0.000000 1011.310527 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.273372] kdmwork-253:40 13414 905136.515453 67646 120 0.000000 755.039354 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.284709] kdmwork-253:41 17980 317215018.006992 831160584 120 0.000000 23949628.934777 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.296053] kdmwork-253:60 18277 317215006.070823 828320819 120 0.000000 26800853.404394 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.307390] kdmwork-253:65 18320 905136.954464 83467 120 0.000000 1010.491550 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.318738] kdmwork-253:97 19159 905136.614986 64446 120 0.000000 735.439791 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.330097] reader#2 50901 317215121.558780 1009535 120 0.000000 316377.996568 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.341431] ptlrpc_hr00_000 64748 20597.461504 3 120 0.000000 0.031058 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.352765] ptlrpc_hr00_001 64749 20597.472384 3 120 0.000000 0.022041 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.364100] ptlrpc_hr00_002 64750 20597.482594 3 120 0.000000 0.019718 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.375435] ptlrpc_hr00_003 64751 20597.493645 3 120 0.000000 0.020779 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.386772] ptlrpc_hr00_004 64752 20597.503293 3 120 0.000000 0.019336 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.398108] ptlrpc_hr00_005 64753 20597.514163 3 120 0.000000 0.021199 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.409456] ptlrpcd_00_10 64815 317214784.096853 4139677 120 0.000000 143851.747560 0.000000 0 / [Tue Mar 17 18:02:12 2020][8592617.420803] ldlm_cb00_001 66875 177900745.975401 8 120 0.000000 0.225604 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.432141] ll_ost00_001 66906 307529537.546910 30980137 120 0.000000 661966.781740 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.443480] ll_ost_io00_000 66946 317215015.966213 52659256 120 0.000000 3669571.730341 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.454814] ll_ost_seq00_00 66976 21569.101574 3 120 0.000000 0.070613 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.466152] ll_ost_seq00_00 66977 21569.115560 3 120 0.000000 0.022904 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.477489] ll_ost_out00_00 67041 21584.641513 3 120 0.000000 0.080182 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.488822] ll_ost_out00_00 67044 21584.658545 3 120 0.000000 0.032901 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.500174] ll_ost_io00_004 68360 317215017.448771 134511822 120 0.000000 9410847.372025 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.511514] ll_ost_io00_008 68377 317214899.024868 112165706 120 0.000000 7849785.830694 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.522859] ll_ost_io00_015 68492 317215011.146492 79774218 120 0.000000 5728877.697591 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.534207] ll_ost00_021 68577 312531912.972123 67920580 120 0.000000 1474730.694478 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.545543] ll_ost00_030 68616 307529537.691604 61169802 120 0.000000 1316874.315299 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.556881] ll_ost00_046 68675 307529537.714173 39865316 120 0.000000 876438.546806 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.568220] ll_ost00_051 68694 317215019.553575 55476427 120 0.000000 1239699.427372 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.579556] ll_ost_io00_019 68698 317214759.988036 67693011 120 0.000000 4970624.346665 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.590893] ll_ost_io00_022 68716 317214845.190451 60827045 120 0.000000 4163247.530875 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.602228] ll_ost00_056 68728 317215130.750700 35427399 120 0.000000 774432.938090 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.613565] ll_ost00_058 68733 307529538.974726 60593350 120 0.000000 1305178.059784 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.624901] ll_ost_io00_027 68756 317214737.916249 159568330 120 0.000000 10031674.179496 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.636243] ll_ost_io00_032 68818 317215015.702370 80485755 120 0.000000 5229723.147474 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.647587] ll_ost_io00_042 68880 317214909.388887 137754902 120 0.000000 9543375.849180 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.658925] ll_ost_io00_044 68886 317215006.679969 144142302 120 0.000000 10046334.762383 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.670260] ll_ost_io00_046 68893 317215011.890091 172792246 120 0.000000 11607827.987809 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.681598] ll_ost_io00_051 68900 317214794.848163 155113003 120 0.000000 9760723.153857 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.692964] ldlm_bl_43 100124 317215122.509412 6105055 120 0.000000 83288.917015 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.704390] ldlm_bl_77 100158 317215122.520282 6113475 120 [8592617.708357] LustreError: dumping log to /tmp/lustre-log.1584493333.68473 [Tue Mar 17 18:02:13 2020] 0.000000 83621.875583 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.722690] ldlm_bl_95 100176 317215122.508600 6107518 120 0.000000 83585.872188 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.734118] ldlm_cn00_016 111456 317215122.744964 19498410 120 0.000000 432409.777546 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.745558] ll_ost_io00_056 87201 317214815.225393 125288952 120 0.000000 7939678.792593 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.756895] ll_ost_io00_062 87562 317214868.605654 71666208 120 0.000000 5196416.105463 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.768235] ll_ost_io00_066 90857 317215016.747401 109877920 120 0.000000 7946459.967255 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.779573] ll_ost_io00_080 40235 317214917.634466 79325922 120 0.000000 5625140.139617 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.790910] ll_ost_io00_083 40238 317214908.397588 102383760 120 0.000000 7550339.906912 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.802264] ll_ost_io00_105 40846 317215006.016239 42421360 120 0.000000 2571023.441342 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.813599] ll_ost_io00_106 40848 317214811.292477 139650764 120 0.000000 9728532.164333 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.824937] ll_ost_io00_108 40851 317214760.555205 89325439 120 0.000000 6032512.041077 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.836276] ldlm_cn00_031 60459 317215041.803316 3602791 120 0.000000 88129.693991 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.847625] ll_ost00_095 85260 317215134.501246 19928572 120 0.000000 439206.453808 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.858963] ll_ost00_097 85264 307529537.536661 39832344 120 0.000000 875514.142079 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.870300] ll_ost00_103 85272 317215048.973370 31128303 120 0.000000 680550.447683 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.881641] kworker/0:1H 92449 317215006.007466 358108178 100 0.000000 2428060.507190 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.892990] kworker/0:1 1566 316811320.086168 2139 120 0.000000 19.319770 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.904327] kworker/0:2 9666 317215122.522276 16502 120 0.000000 396.389564 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.915660] kworker/u256:1 11485 317215122.531643 17439 120 0.000000 680.100742 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.926999] kworker/u256:2 13499 317215114.437547 4049 120 0.000000 105.837662 0.000000 0 / [Tue Mar 17 18:02:13 2020][8592617.938424] [Tue Mar 17 18:02:13 2020][8592617.940102] cpu#1, 1996.233 MHz [Tue Mar 17 18:02:13 2020][8592617.943424] .nr_running : 0 [Tue Mar 17 18:02:13 2020][8592617.948215] .load : 0 [Tue Mar 17 18:02:13 2020][8592617.953007] .nr_switches : 12019554408 [Tue Mar 17 18:02:13 2020][8592617.958681] .nr_load_updates : 2978047060 [Tue Mar 17 18:02:13 2020][8592617.964257] .nr_uninterruptible : -149990 [Tue Mar 17 18:02:13 2020][8592617.969571] .next_balance : 12887.262471 [Tue Mar 17 18:02:13 2020][8592617.975330] .curr->pid : 0 [Tue Mar 17 18:02:13 2020][8592617.980128] .clock : 8592617979.339266 [Tue Mar 17 18:02:13 2020][8592617.986305] .cpu_load[0] : 0 [Tue Mar 17 18:02:13 2020][8592617.991113] .cpu_load[1] : 0 [Tue Mar 17 18:02:13 2020][8592617.995908] .cpu_load[2] : 0 [Tue Mar 17 18:02:13 2020][8592618.000701] .cpu_load[3] : 0 [Tue Mar 17 18:02:13 2020][8592618.005507] .cpu_load[4] : 0 [Tue Mar 17 18:02:13 2020][8592618.010304] .avg_idle : 10000000 [Tue Mar 17 18:02:13 2020][8592618.015718] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:13 2020][8592618.021048] [Tue Mar 17 18:02:13 2020][8592618.022719] cfs_rq[1]:/ [Tue Mar 17 18:02:13 2020][8592618.025346] .exec_clock : 0.000000 [Tue Mar 17 18:02:13 2020][8592618.030745] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:13 2020][8592618.036144] .min_vruntime : 180854200.136104 [Tue Mar 17 18:02:13 2020][8592618.042236] .max_vruntime : 0.000001 [Tue Mar 17 18:02:13 2020][8592618.047636] .spread : 0.000000 [Tue Mar 17 18:02:13 2020][8592618.053037] .spread0 : -136360934.365142 [Tue Mar 17 18:02:13 2020][8592618.059216] .nr_spread_over : 0 [Tue Mar 17 18:02:13 2020][8592618.064007] .nr_running : 0 [Tue Mar 17 18:02:13 2020][8592618.068800] .load : 0 [Tue Mar 17 18:02:13 2020][8592618.073592] .runnable_load_avg : 0 [Tue Mar 17 18:02:13 2020][8592618.078385] .blocked_load_avg : 0 [Tue Mar 17 18:02:13 2020][8592618.083179] .tg_load_avg : 0 [Tue Mar 17 18:02:13 2020][8592618.087972] .tg_load_contrib : 0 [Tue Mar 17 18:02:13 2020][8592618.092763] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:13 2020][8592618.097558] .tg->runnable_avg : 0 [Tue Mar 17 18:02:13 2020][8592618.102350] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:13 2020][8592618.107141] .throttled : 0 [Tue Mar 17 18:02:13 2020][8592618.111933] .throttle_count : 0 [Tue Mar 17 18:02:13 2020][8592618.116727] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:13 2020][8592618.121520] .avg->runnable_avg_period : 47440 [Tue Mar 17 18:02:13 2020][8592618.126663] [Tue Mar 17 18:02:13 2020][8592618.128332] rt_rq[1]:/ [Tue Mar 17 18:02:13 2020][8592618.130871] .rt_nr_running : 0 [Tue Mar 17 18:02:13 2020][8592618.135663] .rt_throttled : 0 [Tue Mar 17 18:02:13 2020][8592618.140456] .rt_time : 0.000000 [Tue Mar 17 18:02:13 2020][8592618.145857] .rt_runtime : 910.829153 [Tue Mar 17 18:02:13 2020][8592618.151428] [Tue Mar 17 18:02:13 2020][8592618.153101] runnable tasks: [Tue Mar 17 18:02:13 2020][8592618.156075] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:13 2020][8592618.167021] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:13 2020][8592618.177884] watchdog/1 13 -2.962399 2148151 0 0.000000 24063.947803 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.189215] migration/1 14 0.000000 72980 0 0.000000 75873.730206 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.200552] ksoftirqd/1 15 180854177.376855 470385207 120 0.000000 9131507.160135 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.211889] kworker/1:0H 17 4463.768606 10 100 0.000000 0.042428 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.223266] kworker/u263:0 4746 1273.053138 3 100 0.000000 0.011752 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.234640] kdmwork-253:10 12599 180854153.509446 821661832 120 0.000000 30203842.968494 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.245995] kworker/1:1H 14342 180854143.760290 944974729 100 0.000000 6361336.291032 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.257336] kdmwork-253:47 18030 1004527.230356 70632 120 0.000000 799.518490 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.268685] kdmwork-253:69 18343 180854155.760222 816428974 120 0.000000 23459701.623763 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.280041] kdmwork-253:118 30891 1004533.266412 62345 120 0.000000 702.566022 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.291383] rpcbind 50250 180854186.210358 286305 120 0.000000 9780.987687 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.302721] agetty 51051 180852943.161275 25 120 0.000000 18.543869 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.314055] ptlrpc_hr01_000 64754 21012.382198 3 120 0.000000 0.035518 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.325386] ptlrpc_hr01_001 64755 21012.396845 3 120 0.000000 0.023704 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.336722] ptlrpc_hr01_002 64756 21012.406233 3 120 0.000000 0.018586 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.348059] ptlrpc_hr01_003 64757 21012.415611 3 120 0.000000 0.019216 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.359394] ptlrpc_hr01_004 64758 21012.424448 3 120 0.000000 0.018716 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.370732] ptlrpc_hr01_005 64759 21012.434748 3 120 0.000000 0.019648 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.382081] ptlrpcd_01_09 64826 180854188.217448 4561567 120 0.000000 136455.202927 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.393422] jbd2/md4-8 66640 180853453.699447 14929770 120 0.000000 2926527.533508 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.404760] ldlm_cb01_001 66878 492592.123910 36 120 0.000000 0.992300 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.416096] ldlm_cn01_001 66887 180854188.143138 10735205 120 0.000000 238982.731899 0.000000 1 / [Tue Mar 17 18:02:13 2020][8592618.427436] ll_ost_seq01_00 66978 22364.492516 3 120 0.000000 0.067258 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.438773] ll_ost_seq01_00 66980 22364.508837 3 120 0.000000 0.025498 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.450113] ll_ost_out01_00 67045 22378.848146 3 120 0.000000 0.094147 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.461447] ll_ost_out01_00 67046 22378.870007 3 120 0.000000 0.031479 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.472785] ll_ost01_004 67825 175057919.273049 22045348 120 0.000000 574534.618847 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.484138] ll_ost01_015 68481 180854186.794947 21183535 120 0.000000 579941.301539 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.495478] ll_ost_io01_017 68509 180853636.488720 55932079 120 0.000000 4070572.146248 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.506835] ll_ost01_049 68666 180854199.159310 20448879 120 0.000000 567129.226684 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.518172] ll_ost01_054 68681 180854196.438751 18496924 120 0.000000 490891.192099 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.529514] ll_ost01_069 68751 180854178.975055 22587216 120 0.000000 640270.434378 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.540851] ll_ost_io01_024 68752 180853499.380829 25735861 120 0.000000 1867656.326540 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.552187] ll_ost_io01_025 68760 180854052.910417 66185727 120 0.000000 5122071.945248 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.563524] ll_ost01_070 68767 180853427.210327 27413600 120 0.000000 717438.647201 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.574868] ll_ost_io01_032 68840 180853573.929157 55612850 120 0.000000 4016671.838188 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.586209] ll_ost_io01_045 68868 180854141.520757 50225757 120 0.000000 3496574.281360 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.597541] ll_ost_io01_046 68873 180853489.576109 84452688 120 0.000000 6927226.272547 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.608877] ll_ost_io01_048 68878 180853739.050133 68896341 120 0.000000 4654216.964110 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.620213] ll_ost_io01_052 68891 180854188.139470 34792557 120 0.000000 2566770.787177 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.631557] ll_ost_io01_055 71821 180853764.062940 105807160 120 0.000000 8027551.833421 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.642895] ll_ost_io01_063 73694 180853465.499602 52901485 120 0.000000 3649210.756216 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.654230] ll_ost_io01_068 73699 180854021.628701 75363266 120 0.000000 4831031.545282 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.665569] ll_ost_io01_081 73712 180853469.461064 26887858 120 0.000000 1968007.216738 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.676901] ll_ost_io01_083 73714 180854009.721577 109843230 120 0.000000 7990547.694450 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.688241] ldlm_cn01_008 73823 180853193.983466 15826208 120 0.000000 337997.391463 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.699587] ldlm_bl_29 100110 180854188.158285 6114297 120 0.000000 83781.465053 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.711004] ldlm_bl_31 100112 180854188.154650 6108264 120 0.000000 83849.699428 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.722443] ldlm_cn01_014 111301 180854171.880963 15204121 120 0.000000 321923.611226 0.000000[8592618.732358] LustreError: dumping log to /tmp/lustre-log.1584493334.68694 [Tue Mar 17 18:02:14 2020] 1 / [Tue Mar 17 18:02:14 2020][8592618.740747] ldlm_cn01_031 111461 180853197.521157 13771131 120 0.000000 285830.805674 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.752210] ll_ost_io01_104 40812 180853752.260573 88272860 120 0.000000 6006579.983197 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.763559] ll_ost01_105 101315 180147860.976513 19681144 120 0.000000 523683.735705 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.774985] ll_ost01_109 101322 180853597.450042 15580532 120 0.000000 371033.630055 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.786405] ll_ost01_111 101326 180854188.161564 23729359 120 0.000000 601864.462836 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.797850] crond 55849 147447848.706327 8 120 0.000000 3.353988 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.809185] kworker/1:0 98441 180854188.163665 1306700 120 0.000000 21732.520675 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.820535] kworker/1:1 13366 180806909.049218 2 120 0.000000 0.008798 0.000000 1 / [Tue Mar 17 18:02:14 2020][8592618.831949] [Tue Mar 17 18:02:14 2020][8592618.833625] cpu#2, 1996.233 MHz [Tue Mar 17 18:02:14 2020][8592618.836946] .nr_running : 0 [Tue Mar 17 18:02:14 2020][8592618.841740] .load : 0 [Tue Mar 17 18:02:14 2020][8592618.846540] .nr_switches : 18365138394 [Tue Mar 17 18:02:14 2020][8592618.852213] .nr_load_updates : 3836878908 [Tue Mar 17 18:02:14 2020][8592618.857788] .nr_uninterruptible : 18704714 [Tue Mar 17 18:02:14 2020][8592618.863189] .next_balance : 12887.263364 [Tue Mar 17 18:02:14 2020][8592618.868961] .curr->pid : 0 [Tue Mar 17 18:02:14 2020][8592618.873753] .clock : 8592618873.342270 [Tue Mar 17 18:02:14 2020][8592618.879932] .cpu_load[0] : 0 [Tue Mar 17 18:02:14 2020][8592618.884751] .cpu_load[1] : 0 [Tue Mar 17 18:02:14 2020][8592618.889543] .cpu_load[2] : 0 [Tue Mar 17 18:02:14 2020][8592618.894335] .cpu_load[3] : 0 [Tue Mar 17 18:02:14 2020][8592618.899130] .cpu_load[4] : 0 [Tue Mar 17 18:02:14 2020][8592618.903945] .avg_idle : 1612194 [Tue Mar 17 18:02:14 2020][8592618.909260] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:14 2020][8592618.914571] [Tue Mar 17 18:02:14 2020][8592618.916243] cfs_rq[2]:/ [Tue Mar 17 18:02:14 2020][8592618.918869] .exec_clock : 0.000000 [Tue Mar 17 18:02:14 2020][8592618.924266] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:14 2020][8592618.929666] .min_vruntime : 212663855.518409 [Tue Mar 17 18:02:14 2020][8592618.935758] .max_vruntime : 0.000001 [Tue Mar 17 18:02:14 2020][8592618.941158] .spread : 0.000000 [Tue Mar 17 18:02:14 2020][8592618.946558] .spread0 : -104551278.982837 [Tue Mar 17 18:02:14 2020][8592618.952736] .nr_spread_over : 0 [Tue Mar 17 18:02:14 2020][8592618.957530] .nr_running : 0 [Tue Mar 17 18:02:14 2020][8592618.962321] .load : 0 [Tue Mar 17 18:02:14 2020][8592618.967115] .runnable_load_avg : 0 [Tue Mar 17 18:02:14 2020][8592618.971908] .blocked_load_avg : 0 [Tue Mar 17 18:02:14 2020][8592618.976701] .tg_load_avg : 0 [Tue Mar 17 18:02:14 2020][8592618.981492] .tg_load_contrib : 0 [Tue Mar 17 18:02:14 2020][8592618.986286] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:14 2020][8592618.991081] .tg->runnable_avg : 0 [Tue Mar 17 18:02:14 2020][8592618.995872] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:14 2020][8592619.000663] .throttled : 0 [Tue Mar 17 18:02:14 2020][8592619.005456] .throttle_count : 0 [Tue Mar 17 18:02:14 2020][8592619.010250] .avg->runnable_avg_sum : 1071 [Tue Mar 17 18:02:14 2020][8592619.015303] .avg->runnable_avg_period : 46343 [Tue Mar 17 18:02:14 2020][8592619.020445] [Tue Mar 17 18:02:14 2020][8592619.022114] rt_rq[2]:/ [Tue Mar 17 18:02:14 2020][8592619.024654] .rt_nr_running : 0 [Tue Mar 17 18:02:14 2020][8592619.029446] .rt_throttled : 0 [Tue Mar 17 18:02:14 2020][8592619.034239] .rt_time : 0.008567 [Tue Mar 17 18:02:14 2020][8592619.039640] .rt_runtime : 928.341561 [Tue Mar 17 18:02:14 2020][8592619.045211] [Tue Mar 17 18:02:14 2020][8592619.046883] runnable tasks: [Tue Mar 17 18:02:14 2020][8592619.049856] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:14 2020][8592619.060803] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:14 2020][8592619.071666] watchdog/2 19 -5.961427 2148153 0 0.000000 24409.354665 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.082998] migration/2 20 0.000000 107637 0 0.000000 189240.905177 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.094334] ksoftirqd/2 21 212663626.864868 445051810 120 0.000000 6339110.747226 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.105670] kworker/2:0H 23 5533.821180 10 100 0.000000 0.030158 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.117045] irq/28-AMD-Vi 270 0.000000 3 49 0.000000 0.031278 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.128385] kworker/2:1H 4773 212663843.443373 1228752255 100 0.000000 6895611.245371 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.139885] kdmwork-253:90 18681 212663843.566508 803506108 120 0.000000 28995359.580058 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.151241] dsm_sa_eventmgr 50187 34806.189724 25 120 0.000000 3.092374 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.162574] dsm_sa_eventmgr 50228 212663843.786536 5644412 120 0.000000 373325.369705 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.173917] rs:main Q:Reg 50893 212663846.185459 957971 120 0.000000 43699.270822 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.185251] reader#3 50902 212663846.127031 1501864 120 0.000000 259485.849821 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.196588] ptlrpc_hr02_001 64761 36783.345697 3 120 0.000000 0.043743 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.207925] kiblnd_sd_02_03 64793 212663850.342187 11166781956 120 0.000000 87369284.442902 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.219443] md2_raid6 66282 212662809.095919 1316559229 120 0.000000 426368052.579771 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.230862] kmmpd-md4 66626 212662930.720879 17176395 120 0.000000 137571.685145 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.242205] ll_ost02_001 66919 212663361.653834 43613074 120 0.000000 1244405.378371 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.253542] ll_ost_create02 66938 211050654.088799 3630832 120 0.000000 64200.927985 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.264880] ll_ost_seq02_00 66984 37584.623241 3 120 0.000000 0.040397 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.276214] ll_ost_out02_00 67047 163632796.995721 8752 120 0.000000 279.008842 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.287559] ll_ost02_010 68342 208440271.709721 34693467 120 0.000000 982731.094451 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.298900] ll_ost_io02_006 68363 212662845.919009 147171694 120 0.000000 10129083.174238 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.310241] ll_ost02_019 68483 212663581.936865 42874697 120 0.000000 1186043.902555 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.321591] ll_ost02_028 68571 212663587.287727 28805830 120 0.000000 809088.974339 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.332929] ll_ost02_029 68574 212663554.759920 52894226 120 0.000000 1478550.267589 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.344273] ll_ost02_043 68639 212663626.934209 29127115 120 0.000000 864356.042985 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.355612] ll_ost02_045 68649 212663833.010481 32434056 120 0.000000 896129.468877 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.366952] ll_ost02_056 68699 211312812.110049 41338199 120 0.000000 1182595.867748 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.378287] ll_ost02_063 68743 208671863.201883 45019364 120 0.000000 1287289.167544 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.389620] ll_ost02_065 68755 212663774.407905 22982820 120 0.000000 622716.654580 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.400956] ll_ost_io02_030 68777 212663250.484794 120848098 120 0.000000 7847208.898156 0.000000 2 / [Tue Mar 17 18:02:14 2020][8592619.412296] ll_ost02_076 68798 208440271.712155 42899325 120 0.000000 1251982.792713 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.423632] ll_ost_io02_041 68849 212663354.105305 92502038 120 0.000000 6273412.011311 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.434963] ll_ost_io02_042 68850 212663135.979736 99623452 120 0.000000 6875920.426906 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.446307] ll_ost_create02 68906 211050654.032733 3915299 120 0.000000 74248.841299 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.457665] ll_ost02_084 98675 212663829.975271 55487481 120 0.000000 1601572.152841 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.468997] ll_ost02_095 98686 212663255.145123 41570698 120 0.000000 1155612.671435 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.480334] ll_ost02_098 98689 212663706.182333 45161833 120 0.000000 1206864.802341 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.491669] ll_ost_io02_059 98930 212663144.855741 187743379 120 0.000000 12760137.771078 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.503007] ll_ost_create02 98979 212663337.448215 5098139 120 0.000000 96613.928285 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.514344] ldlm_bl_24 100105 212663843.530131 6108046 120 0.000000 83865.626815 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.525772] ldlm_bl_68 100149 212663843.526785 6111514 120 0.000000 83847.900640 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.537220] ll_ost_io02_067 87195 212663134.519688 68160301 120 0.000000 4534101.908284 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.548564] ll_ost_io02_075 90846 212663124.195362 123566622 120 0.000000 8175890.926443 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.559906] ll_ost_io02_084 90864 212662845.109147 91341195 120 0.000000 5933083.075707 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.571241] ll_ost_io02_090 90877 212663515.279221 123495224 120 0.000000 8206329.574452 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.582583] ll_ost_io02_096 40783 212663454.893879 103719759 120 0.000000 7176602.130477 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.593919] ll_ost_io02_097 40784 212663177.805626 143721697 120 0.000000 9294915.052818 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.605265] ll_ost_io02_111 40842 212663128.054658 118231994 120 0.000000 8203882.567179 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.616610] ll_ost02_102 101330 212663520.616190 35643518 120 0.000000 1009975.350099 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.628032] ll_ost02_104 101332 211312812.110460 37271685 120 0.000000 1105521.151942 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.639472] ll_ost_out02_00 105890 163630815.339116 8685 120 0.000000 162.849966 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.650905] kworker/2:2 627 212663843.550191 207932 120 0.000000 1314.504955 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.662246] kworker/2:0 10361 212511872.273596 2 120 0.000000 0.008787 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.673591] kworker/u259:4 13962 212663730.300177 12 120 0.000000 0.130566 0.000000 2 / [Tue Mar 17 18:02:15 2020][8592619.685003] [Tue Mar 17 18:02:15 2020][8592619.686673] cpu#3, 1996.233 MHz [Tue Mar 17 18:02:15 2020][8592619.689995] .nr_running : 0 [Tue Mar 17 18:02:15 2020][8592619.694787] .load : 0 [Tue Mar 17 18:02:15 2020][8592619.699580] .nr_switches : 14026244328 [Tue Mar 17 18:02:15 2020][8592619.705252] .nr_load_updates : 3316403908 [Tue Mar 17 18:02:15 2020][8592619.710828] .nr_uninterruptible : -172497 [Tue Mar 17 18:02:15 2020][8592619.716143] .next_balance : 12887.264217 [Tue Mar 17 18:02:15 2020][8592619.721901] .curr->pid : 0 [Tue Mar 17 18:02:15 2020][8592619.726697] .clock : 8592619726.348617 [Tue Mar 17 18:02:15 2020][8592619.732877] .cpu_load[0] : 0 [Tue Mar 17 18:02:15 2020][8592619.737683] .cpu_load[1] : 0 [Tue Mar 17 18:02:15 2020][8592619.742479] .cpu_load[2] : 0 [Tue Mar 17 18:02:15 2020][8592619.747272] .cpu_load[3] : 0 [Tue Mar 17 18:02:15 2020][8592619.752065] .cpu_load[4] : 0 [Tue Mar 17 18:02:15 2020][8592619.756872] .avg_idle : 10000000 [Tue Mar 17 18:02:15 2020][8592619.762288] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:15 2020][8592619.767603] [Tue Mar 17 18:02:15 2020][8592619.769274] cfs_rq[3]:/ [Tue Mar 17 18:02:15 2020][8592619.771900] .exec_clock : 0.000000 [Tue Mar 17 18:02:15 2020][8592619.777298] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:15 2020][8592619.782699] .min_vruntime : 190458872.808285 [Tue Mar 17 18:02:15 2020][8592619.788790] .max_vruntime : 0.000001 [Tue Mar 17 18:02:15 2020][8592619.794191] .spread : 0.000000 [Tue Mar 17 18:02:15 2020][8592619.799590] .spread0 : -126756261.692961 [Tue Mar 17 18:02:15 2020][8592619.805770] .nr_spread_over : 0 [Tue Mar 17 18:02:15 2020][8592619.810561] .nr_running : 0 [Tue Mar 17 18:02:15 2020][8592619.815353] .load : 0 [Tue Mar 17 18:02:15 2020][8592619.820146] .runnable_load_avg : 0 [Tue Mar 17 18:02:15 2020][8592619.824940] .blocked_load_avg : 0 [Tue Mar 17 18:02:15 2020][8592619.829732] .tg_load_avg : 0 [Tue Mar 17 18:02:15 2020][8592619.834526] .tg_load_contrib : 0 [Tue Mar 17 18:02:15 2020][8592619.839319] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:15 2020][8592619.844111] .tg->runnable_avg : 0 [Tue Mar 17 18:02:15 2020][8592619.848903] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:15 2020][8592619.853696] .throttled : 0 [Tue Mar 17 18:02:15 2020][8592619.858488] .throttle_count : 0 [Tue Mar 17 18:02:15 2020][8592619.863280] .avg->runnable_avg_sum : 2 [Tue Mar 17 18:02:15 2020][8592619.868073] .avg->runnable_avg_period : 47607 [Tue Mar 17 18:02:15 2020][8592619.873215] [Tue Mar 17 18:02:15 2020][8592619.874885] rt_rq[3]:/ [Tue Mar 17 18:02:15 2020][8592619.877425] .rt_nr_running : 0 [Tue Mar 17 18:02:15 2020][8592619.882218] .rt_throttled : 0 [Tue Mar 17 18:02:15 2020][8592619.887011] .rt_time : 0.000000 [Tue Mar 17 18:02:15 2020][8592619.892410] .rt_runtime : 950.000000 [Tue Mar 17 18:02:15 2020][8592619.897983] [Tue Mar 17 18:02:15 2020][8592619.899654] runnable tasks: [Tue Mar 17 18:02:15 2020][8592619.902629] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:15 2020][8592619.913574] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:15 2020][8592619.924437] watchdog/3 25 -5.960035 2148152 0 0.000000 23389.790664 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.935770] migration/3 26 0.000000 64461 0 0.000000 95061.977090 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.947105] ksoftirqd/3 27 190458672.136570 550734455 120 0.000000 9473008.217752 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.958442] kworker/3:0H 29 977.634557 10 100 0.000000 0.036297 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.969814] irq/29-AMD-Vi 271 0.000000 3 49 0.000000 0.036670 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.981154] scsi_tmf_2 2676 451.964052 2 100 0.000000 0.008998 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592619.992487] kworker/u265:0 4748 963.197333 3 100 0.000000 0.011060 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.003826] kworker/3:1H 5605 190458759.083364 1035563332 100 0.000000 6422058.181846 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.015289] kdmwork-253:32 12715 190458731.349410 870000727 120 0.000000 30777781.984386 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.026636] kdmwork-253:46 18020 523059.059661 226952 120 0.000000 3115.284085 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.037977] kdmwork-253:59 18268 190458681.183266 874570239 120 0.000000 24999865.919452 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.049316] kdmwork-253:64 18301 523058.288857 63264 120 0.000000 729.538081 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.060656] kdmwork-253:85 18562 523058.277448 63783 120 0.000000 742.376848 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.072013] mdadm 50192 190458733.995785 10917 120 0.000000 24981.061681 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.083356] ptlrpc_hr03_000 64766 25378.629818 3 120 0.000000 0.036227 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.094691] ptlrpc_hr03_001 64767 25378.634327 3 120 0.000000 0.015529 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.106026] ptlrpc_hr03_002 64768 25378.643134 3 120 0.000000 0.019077 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.117362] ptlrpc_hr03_003 64769 25378.653112 3 120 0.000000 0.019646 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.128700] ptlrpc_hr03_004 64770 25378.662690 3 120 0.000000 0.019186 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.140034] ptlrpc_hr03_005 64771 25378.673741 3 120 0.000000 0.020268 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.151375] kiblnd_sd_03_01 64795 190458873.015295 9107952576 120 0.000000 69269820.843839 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.162802] ptlrpcd_03_01 64842 190458080.968860 3567637 120 0.000000 115920.715086 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.174148] ldlm_cb03_000 66881 26211.910736 3 120 0.000000 0.055335 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.185483] ldlm_cb03_001 66882 26211.929872 3 120 0.000000 0.028113 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.196823] ldlm_cn03_001 66893 190458730.162020 13783980 120 0.000000 306378.621954 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.208162] ll_ost_seq03_00 66992 26221.065779 3 120 0.000000 0.078197 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.219495] ll_ost03_003 67232 190458686.758124 17452233 120 0.000000 467001.293689 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.230833] ll_ost03_004 67857 190458619.954109 19460251 120 0.000000 588730.871430 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.242169] ll_ost03_007 68344 190458676.497905 12479826 120 0.000000 382565.002248 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.253504] ll_ost03_011 68361 187529034.713030 26771519 120 0.000000 758790.397884 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.264856] ll_ost03_016 68539 190458673.950209 21554684 120 0.000000 628692.724246 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.276192] ll_ost03_021 68548 190458674.553358 23878605 120 0.000000 648824.595289 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.287538] ll_ost03_038 68614 190458807.073157 23719290 120 0.000000 687397.241391 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.298872] ll_ost03_041 68625 180875368.572368 19480070 120 0.000000 534644.855034 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.310209] ll_ost03_044 68636 190458684.001267 16858421 120 0.000000 571702.908819 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.321546] ll_ost03_051 68661 190458829.336985 10625180 120 0.000000 327701.134666 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.332878] ll_ost03_052 68663 190458695.579438 14384322 120 0.000000 451216.849396 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.344216] ll_ost03_054 68673 187529034.712409 23579147 120 0.000000 669417.359382 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.355555] ll_ost03_059 68700 190458572.221083 18246707 120 0.000000 523248.149052 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.366886] ll_ost_io03_019 68702 190458123.354796 128441847 120 0.000000 7685271.122491 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.378223] ll_ost_io03_021 68714 190458475.792078 96771362 120 0.000000 5866171.230279 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.389564] ll_ost03_064 68753 187529034.712530 17946392 120 0.000000 520221.654746 0.000000 3 / [Tue Mar 17 18:02:15 2020][8592620.400909] ll_ost_io03_037 68877 190458245.980286 11915204 120 0.000000 675297.949230 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.412240] ll_ost_io03_041 68888 190458673.663990 63097013 120 0.000000 3631080.682223 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.423606] ldlm_bl_32 100113 190458858.754790 6111526 120 0.000000 83863.531384 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.435055] ll_ost_io03_049 84742 190458676.295642 68322210 120 0.000000 4137934.411520 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.446388] ll_ost_io03_054 87187 190458673.354957 22983448 120 0.000000 1544593.395913 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.457731] ll_ost_io03_066 90852 190458265.469750 102021261 120 0.000000 6002288.333829 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.469069] ll_ost_io03_070 90861 190458681.624102 59918238 120 0.000000 3705247.257400 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.480414] ll_ost_io03_075 40250 190458631.268308 108100195 120 0.000000 6297675.765593 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.491750] ll_ost_io03_078 40773 190458525.178797 114673582 120 0.000000 7310652.146630 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.503085] ll_ost_io03_079 40774 190458081.138115 96522175 120 0.000000 6006629.892517 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.514422] ll_ost_io03_080 40775 190458498.527849 49239356 120 0.000000 2789678.522599 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.525763] ll_ost_io03_082 40791 190458672.234285 87527824 120 0.000000 5091422.875677 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.537106] ll_ost_io03_108 40861 190458324.030893 100747587 120 0.000000 5887409.818191 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.548449] ll_ost03_074 70320 190458676.956184 4705921 120 0.000000 157294.566729 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.559787] ll_ost03_097 74618 190458693.025130 12422108 120 0.000000 354830.920766 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.571122] ll_ost03_099 90274 190458677.903090 18157400 120 0.000000 513293.263089 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.582455] ll_ost03_104 125737 187529034.712810 10162578 120 0.000000 296475.910665 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.593879] ll_ost03_109 125742 190458678.847948 11569520 120 0.000000 327697.498959 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.605301] ll_ost_create03 6305 188383983.689451 1059332 120 0.000000 17981.795962 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.616637] ll_ost_out03_00 107886 152037295.117876 1509032 120 0.000000 36974.554478 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.628064] kworker/3:2 37556 190458860.582172 919790 120 0.000000 16195.768039 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.639404] kworker/3:1 4448 190095847.489642 2 120 0.000000 0.010300 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.650752] kworker/u260:1 13948 190452374.423017 2 120 0.000000 0.010771 0.000000 3 / [Tue Mar 17 18:02:16 2020][8592620.662126] [Tue Mar 17 18:02:16 2020][8592620.663803] cpu#4, 1996.233 MHz [Tue Mar 17 18:02:16 2020][8592620.667126] .nr_running : 0 [Tue Mar 17 18:02:16 2020][8592620.671927] .load : 0 [Tue Mar 17 18:02:16 2020][8592620.676718] .nr_switches : 17303493865 [Tue Mar 17 18:02:16 2020][8592620.682379] .nr_load_updates : 3432065114 [Tue Mar 17 18:02:16 2020][8592620.687965] .nr_uninterruptible : -111439 [Tue Mar 17 18:02:16 2020][8592620.693280] .next_balance : 12887.265194 [Tue Mar 17 18:02:16 2020][8592620.699026] .curr->pid : 0 [Tue Mar 17 18:02:16 2020][8592620.703833] .clock : 8592620703.351823 [Tue Mar 17 18:02:16 2020][8592620.710016] .cpu_load[0] : 0 [Tue Mar 17 18:02:16 2020][8592620.714807] .cpu_load[1] : 0 [Tue Mar 17 18:02:16 2020][8592620.719615] .cpu_load[2] : 0 [Tue Mar 17 18:02:16 2020][8592620.724410] .cpu_load[3] : 0 [Tue Mar 17 18:02:16 2020][8592620.729203] .cpu_load[4] : 0 [Tue Mar 17 18:02:16 2020][8592620.733997] .avg_idle : 2028255 [Tue Mar 17 18:02:16 2020][8592620.739325] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:16 2020][8592620.744640] [Tue Mar 17 18:02:16 2020][8592620.746309] cfs_rq[4]:/ [Tue Mar 17 18:02:16 2020][8592620.748933] .exec_clock : 0.000000 [Tue Mar 17 18:02:16 2020][8592620.754334] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:16 2020][8592620.759732] .min_vruntime : 329699982.543547 [Tue Mar 17 18:02:16 2020][8592620.765826] .max_vruntime : 0.000001 [Tue Mar 17 18:02:16 2020][8592620.771226] .spread : 0.000000 [Tue Mar 17 18:02:16 2020][8592620.776623] .spread0 : 12484848.042301 [Tue Mar 17 18:02:16 2020][8592620.782632] .nr_spread_over : 0 [Tue Mar 17 18:02:16 2020][8592620.787423] .nr_running : 0 [Tue Mar 17 18:02:16 2020][8592620.792215] .load : 0 [Tue Mar 17 18:02:16 2020][8592620.797009] .runnable_load_avg : 0 [Tue Mar 17 18:02:16 2020][8592620.801800] .blocked_load_avg : 0 [Tue Mar 17 18:02:16 2020][8592620.806594] .tg_load_avg : 0 [Tue Mar 17 18:02:16 2020][8592620.811385] .tg_load_contrib : 0 [Tue Mar 17 18:02:16 2020][8592620.816178] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:16 2020][8592620.820971] .tg->runnable_avg : 0 [Tue Mar 17 18:02:16 2020][8592620.825763] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:16 2020][8592620.830556] .throttled : 0 [Tue Mar 17 18:02:16 2020][8592620.835351] .throttle_count : 0 [Tue Mar 17 18:02:16 2020][8592620.840144] .avg->runnable_avg_sum : 58 [Tue Mar 17 18:02:16 2020][8592620.845023] .avg->runnable_avg_period : 46709 [Tue Mar 17 18:02:16 2020][8592620.850165] [Tue Mar 17 18:02:16 2020][8592620.851835] rt_rq[4]:/ [Tue Mar 17 18:02:16 2020][8592620.854374] .rt_nr_running : 0 [Tue Mar 17 18:02:16 2020][8592620.859165] .rt_throttled : 0 [Tue Mar 17 18:02:16 2020][8592620.863958] .rt_time : 0.000000 [Tue Mar 17 18:02:16 2020][8592620.869359] .rt_runtime : 950.000000 [Tue Mar 17 18:02:16 2020][8592620.874929] [Tue Mar 17 18:02:16 2020][8592620.876603] runnable tasks: [Tue Mar 17 18:02:16 2020][8592620.879576] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:16 2020][8592620.890523] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:16 2020][8592620.901384] kthreadd 2 329699872.840899 178088 120 0.000000 6875.088959 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.912719] rcu_sched 10 329699982.742437 1598707819 120 0.000000 20738871.852599 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.924143] watchdog/4 31 -8.971787 2148154 0 0.000000 22547.955277 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.935475] migration/4 32 0.000000 117386 0 0.000000 109942.048177 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.946813] ksoftirqd/4 33 329699870.146153 598090257 120 0.000000 10969525.365193 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.958147] kworker/4:0H 35 4081.768566 10 100 0.000000 0.036119 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.969557] kdmwork-253:5 12514 329699736.833604 824103951 120 0.000000 30279904.837481 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.980915] kworker/4:1H 14341 329699724.833733 1159567778 100 0.000000 7815683.626961 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592620.992353] kdmwork-253:55 18192 550901.715941 227488 120 0.000000 3152.022799 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.003700] kdmwork-253:95 19071 329699724.857149 829781570 120 0.000000 28081768.154355 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.015049] kdmwork-253:112 25379 329699736.820178 813724759 120 0.000000 30766376.174952 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.026397] kiblnd_sd_00_00 64782 329699982.891215 11543852096 120 0.000000 74180987.312621 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.037908] kiblnd_sd_00_01 64783 329699971.361159 11544583186 120 0.000000 74107949.324858 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.049423] ptlrpcd_00_03 64808 329699353.044111 4229945 120 0.000000 138848.406021 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.060791] ll_ost_io00_011 68387 329699570.910582 114311270 120 0.000000 7400030.585263 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.072124] ll_ost_io00_013 68389 329699663.266536 81513920 120 0.000000 5661198.945096 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.083469] ll_ost_io00_014 68488 329699338.933935 110400102 120 0.000000 7144878.983243 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.094823] ll_ost00_031 68621 329699325.946964 68742551 120 0.000000 1459489.587025 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.106162] ll_ost00_036 68638 319204398.334870 58624269 120 0.000000 1269825.505782 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.117499] ll_ost00_044 68668 319204398.433404 38394213 120 0.000000 853577.239689 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.128837] ll_ost_io00_023 68721 329699417.490758 145970438 120 0.000000 8778227.525271 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.140173] ll_ost00_062 68769 319204398.440838 61029549 120 0.000000 1353604.908037 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.151507] ll_ost00_064 68785 319204398.330369 65665774 120 0.000000 1385904.089711 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.162843] ll_ost00_072 68813 319204398.349135 35185110 120 0.000000 782728.941112 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.174184] ldlm_cn00_004 68909 329698991.581451 32860193 120 0.000000 775025.963452 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.185546] ldlm_bl_13 100094 329699877.843137 6114373 120 0.000000 148555.766523 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.196972] ldlm_bl_30 100111 329699879.974328 6111750 120 0.000000 83661.069680 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.208402] ldlm_bl_79 100160 329699969.909750 6114627 120 0.000000 83845.410602 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.219834] ldlm_cn00_022 30586 329698983.809158 38787015 120 0.000000 882058.094719 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.231190] ll_ost_io00_071 90878 329699565.832833 94322398 120 0.000000 6629025.304956 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.242526] ll_ost_io00_081 40236 329699565.492516 137167614 120 0.000000 9598772.050399 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.253886] ll_ost00_088 101339 319204398.561916 53506153 120 0.000000 1109754.036613 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.265315] ll_ost00_093 85258 319204398.413747 41234156 120 0.000000 926049.657117 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.276654] ll_ost00_099 85268 325224430.044252 10798210 120 0.000000 231220.108620 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.288003] kworker/4:0 11825 329699968.884162 398517 120 0.000000 8497.819295 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.299343] kworker/4:1 90682 327764013.510787 2 120 0.000000 0.010419 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.310685] kworker/u257:0 11538 329699980.364530 4471 120 0.000000 266.944276 0.000000 0 / [Tue Mar 17 18:02:16 2020][8592621.322054] [Tue Mar 17 18:02:16 2020][8592621.323730] cpu#5, 1996.233 MHz [Tue Mar 17 18:02:16 2020][8592621.327052] .nr_running : 0 [Tue Mar 17 18:02:16 2020][8592621.331846] .load : 0 [Tue Mar 17 18:02:16 2020][8592621.336648] .nr_switches : 11730956650 [Tue Mar 17 18:02:16 2020][8592621.342313] .nr_load_updates : 2918882866 [Tue Mar 17 18:02:16 2020][8592621.347902] .nr_uninterruptible : -156899 [Tue Mar 17 18:02:16 2020][8592621.353217] .next_balance : 12887.265856 [Tue Mar 17 18:02:16 2020][8592621.358963] .curr->pid : 0 [Tue Mar 17 18:02:16 2020][8592621.363771] .clock : 8592621363.353556 [Tue Mar 17 18:02:16 2020][8592621.369951] .cpu_load[0] : 0 [Tue Mar 17 18:02:16 2020][8592621.374746] .cpu_load[1] : 0 [Tue Mar 17 18:02:16 2020][8592621.379560] .cpu_load[2] : 0 [Tue Mar 17 18:02:16 2020][8592621.384358] .cpu_load[3] : 0 [Tue Mar 17 18:02:16 2020][8592621.389148] .cpu_load[4] : 0 [Tue Mar 17 18:02:16 2020][8592621.393942] .avg_idle : 6374152 [Tue Mar 17 18:02:16 2020][8592621.399285] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:16 2020][8592621.404603] [Tue Mar 17 18:02:16 2020][8592621.406271] cfs_rq[5]:/ [Tue Mar 17 18:02:16 2020][8592621.408897] .exec_clock : 0.000000 [Tue Mar 17 18:02:17 2020][8592621.414295] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:17 2020][8592621.419694] .min_vruntime : 175158213.599967 [Tue Mar 17 18:02:17 2020][8592621.425788] .max_vruntime : 0.000001 [Tue Mar 17 18:02:17 2020][8592621.431187] .spread : 0.000000 [Tue Mar 17 18:02:17 2020][8592621.436588] .spread0 : -142056920.901279 [Tue Mar 17 18:02:17 2020][8592621.442766] .nr_spread_over : 0 [Tue Mar 17 18:02:17 2020][8592621.447559] .nr_running : 0 [Tue Mar 17 18:02:17 2020][8592621.452353] .load : 0 [Tue Mar 17 18:02:17 2020][8592621.457143] .runnable_load_avg : 0 [Tue Mar 17 18:02:17 2020][8592621.461936] .blocked_load_avg : 0 [Tue Mar 17 18:02:17 2020][8592621.466731] .tg_load_avg : 0 [Tue Mar 17 18:02:17 2020][8592621.471522] .tg_load_contrib : 0 [Tue Mar 17 18:02:17 2020][8592621.476316] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:17 2020][8592621.481107] .tg->runnable_avg : 0 [Tue Mar 17 18:02:17 2020][8592621.485900] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:17 2020][8592621.490694] .throttled : 0 [Tue Mar 17 18:02:17 2020][8592621.495486] .throttle_count : 0 [Tue Mar 17 18:02:17 2020][8592621.500278] .avg->runnable_avg_sum : 13 [Tue Mar 17 18:02:17 2020][8592621.505159] .avg->runnable_avg_period : 47698 [Tue Mar 17 18:02:17 2020][8592621.510301] [Tue Mar 17 18:02:17 2020][8592621.511970] rt_rq[5]:/ [Tue Mar 17 18:02:17 2020][8592621.514509] .rt_nr_running : 0 [Tue Mar 17 18:02:17 2020][8592621.519302] .rt_throttled : 0 [Tue Mar 17 18:02:17 2020][8592621.524095] .rt_time : 0.000000 [Tue Mar 17 18:02:17 2020][8592621.529493] .rt_runtime : 950.000000 [Tue Mar 17 18:02:17 2020][8592621.535066] [Tue Mar 17 18:02:17 2020][8592621.536740] runnable tasks: [Tue Mar 17 18:02:17 2020][8592621.539712] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:17 2020][8592621.550659] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:17 2020][8592621.561525] watchdog/5 36 -8.959674 2148152 0 0.000000 23117.731590 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.572861] migration/5 37 0.000000 70601 0 0.000000 75544.249717 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.584199] ksoftirqd/5 38 175158120.037392 462645751 120 0.000000 8999658.251222 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.595533] kworker/5:0H 40 4473.619701 10 100 0.000000 0.043612 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.606956] kdmwork-253:43 18003 175158069.961286 873752515 120 0.000000 29954457.966279 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.618300] kworker/5:1H 18103 175158061.881415 931688113 100 0.000000 6204899.729247 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.629655] kdmwork-253:92 18701 175158061.981624 829988443 120 0.000000 29487661.797378 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.641020] kiblnd_sd_01_01 64787 175158213.706158 6692971494 120 0.000000 50980136.489880 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.652444] ptlrpcd_rcv 64804 104025668.336664 10276 120 0.000000 912.884309 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.663808] ll_ost_io01_003 68349 175157875.728188 58198800 120 0.000000 3800782.687084 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.675144] ll_ost_io01_009 68380 175157265.096879 66657716 120 0.000000 4635535.123138 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.686495] ll_ost_io01_015 68502 175158064.314064 61731626 120 0.000000 4787238.591724 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.697838] ll_ost01_023 68568 175157296.548669 22728558 120 0.000000 549516.160589 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.709182] ll_ost01_041 68631 175158027.671014 20802877 120 0.000000 514618.884443 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.720523] ll_ost01_052 68676 175157367.121391 13982530 120 0.000000 374200.839901 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.731857] ll_ost01_059 68697 169495617.059180 27874092 120 0.000000 724813.314600 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.743194] ll_ost01_064 68731 169495617.055934 28091907 120 0.000000 739298.552423 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.754526] ll_ost01_068 68741 169495617.037459 21669516 120 0.000000 532126.382081 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.765868] ll_ost_io01_028 68786 175158065.031079 41632400 120 0.000000 2968582.257654 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.777214] ll_ost_io01_037 68853 175158065.050004 49318727 120 0.000000 3659343.490534 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.788552] ll_ost_io01_041 68863 175157362.808711 74765336 120 0.000000 5123363.311746 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.799889] ll_ost_io01_042 68865 175158069.120632 95829238 120 0.000000 7088865.816011 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.811227] ll_ost_io01_051 68889 175157218.447160 54422996 120 0.000000 4426667.386353 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.822563] ldlm_cn01_005 68912 175156911.966254 16412258 120 0.000000 346922.957084 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.833899] ldlm_bl_07 70500 175158201.257453 6338060 120 0.000000 87633.198873 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.845234] ll_ost_io01_058 71824 175157373.990518 40824710 120 0.000000 2434331.086653 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.856568] ll_ost_io01_061 71827 175158061.918838 79403963 120 0.000000 5135565.905227 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.867904] ll_ost_io01_066 73697 175157275.356817 76409756 120 0.000000 5761095.060478 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.879240] ll_ost_io01_069 73700 175157378.858325 94091243 120 0.000000 6915191.861292 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.890578] ll_ost_io01_073 73704 175157373.913093 45258559 120 0.000000 3434669.980399 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.901917] ll_ost_io01_087 73718 175157301.412380 107413017 120 0.000000 8086713.546879 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.913252] ll_ost01_085 97759 169495617.070942 18286755 120 0.000000 519666.784044 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.924595] ldlm_bl_25 100106 175158198.019344 6107732 120 0.000000 83712.614895 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.936025] ldlm_bl_64 100145 175158201.424798 6113995 120 0.000000 83982.554565 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.947454] ldlm_bl_99 100180 175158201.909161 6108942 120 0.000000 83774.746178 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.958880] ldlm_cn01_013 111300 175156934.665409 16809840 120 0.000000 349984.583837 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.970341] ll_ost_io01_095 40782 175157876.686661 46334099 120 0.000000 3080585.629990 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.981677] ll_ost_io01_103 40811 175157207.660115 79989155 120 0.000000 5689297.694397 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592621.993008] ll_ost_io01_105 40816 175158061.940267 58550996 120 0.000000 4283570.346826 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.004345] ll_ost_io01_107 40820 175157374.759899 56331015 120 0.000000 3703902.934150 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.015681] ll_ost_io01_108 40825 175158073.837545 32809886 120 0.000000 2370231.104207 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.027050] crond 64012 126163515.310825 6 120 0.000000 3.387520 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.038390] kworker/5:1 24319 175158191.831917 2223031 120 0.000000 43418.469511 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.049740] kworker/5:2 12197 175073010.417453 3 120 0.000000 0.019446 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.061078] kworker/u258:2 13341 175158151.652451 288 120 0.000000 165.624132 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.072417] cleanup 14288 175158201.562059 10 120 0.000000 5.960517 0.000000 1 / [Tue Mar 17 18:02:17 2020][8592622.083791] [Tue Mar 17 18:02:17 2020][8592622.085467] cpu#6, 1996.233 MHz [Tue Mar 17 18:02:17 2020][8592622.088789] .nr_running : 6 [Tue Mar 17 18:02:17 2020][8592622.093580] .load : 91833 [Tue Mar 17 18:02:17 2020][8592622.098721] .nr_switches : 17557483770 [Tue Mar 17 18:02:17 2020][8592622.104394] .nr_load_updates : 3735900014 [Tue Mar 17 18:02:17 2020][8592622.109969] .nr_uninterruptible : 18287990 [Tue Mar 17 18:02:17 2020][8592622.115370] .next_balance : 12887.266711 [Tue Mar 17 18:02:17 2020][8592622.121129] .curr->pid : 14041 [Tue Mar 17 18:02:17 2020][8592622.126271] .clock : 8592364080.340222 [Tue Mar 17 18:02:17 2020][8592622.132450] .cpu_load[0] : 91833 [Tue Mar 17 18:02:17 2020][8592622.137606] .cpu_load[1] : 91833 [Tue Mar 17 18:02:17 2020][8592622.142746] .cpu_load[2] : 91833 [Tue Mar 17 18:02:17 2020][8592622.147887] .cpu_load[3] : 91833 [Tue Mar 17 18:02:17 2020][8592622.153041] .cpu_load[4] : 91833 [Tue Mar 17 18:02:17 2020][8592622.158184] .avg_idle : 4413875 [Tue Mar 17 18:02:17 2020][8592622.163495] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:17 2020][8592622.168821] [Tue Mar 17 18:02:17 2020][8592622.170496] cfs_rq[6]:/ [Tue Mar 17 18:02:17 2020][8592622.173121] .exec_clock : 0.000000 [Tue Mar 17 18:02:17 2020][8592622.178521] .MIN_vruntime : 206642737.066285 [Tue Mar 17 18:02:17 2020][8592622.184612] .min_vruntime : 206642749.066285 [Tue Mar 17 18:02:17 2020][8592622.190706] .max_vruntime : 206642737.066390 [Tue Mar 17 18:02:17 2020][8592622.196798] .spread : 0.000105 [Tue Mar 17 18:02:17 2020][8592622.202196] .spread0 : -110572386.448668 [Tue Mar 17 18:02:17 2020][8592622.208375] .nr_spread_over : 0 [Tue Mar 17 18:02:17 2020][8592622.213169] .nr_running : 4 [Tue Mar 17 18:02:17 2020][8592622.217960] .load : 91833 [Tue Mar 17 18:02:17 2020][8592622.223101] .runnable_load_avg : 193 [Tue Mar 17 18:02:17 2020][8592622.228066] .blocked_load_avg : 0 [Tue Mar 17 18:02:17 2020][8592622.232860] .tg_load_avg : 0 [Tue Mar 17 18:02:17 2020][8592622.237653] .tg_load_contrib : 0 [Tue Mar 17 18:02:17 2020][8592622.242444] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:17 2020][8592622.247237] .tg->runnable_avg : 0 [Tue Mar 17 18:02:17 2020][8592622.252029] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:17 2020][8592622.256823] .throttled : 0 [Tue Mar 17 18:02:17 2020][8592622.261616] .throttle_count : 0 [Tue Mar 17 18:02:17 2020][8592622.266408] .avg->runnable_avg_sum : 9391 [Tue Mar 17 18:02:17 2020][8592622.271461] .avg->runnable_avg_period : 47243 [Tue Mar 17 18:02:17 2020][8592622.276605] [Tue Mar 17 18:02:17 2020][8592622.278273] rt_rq[6]:/ [Tue Mar 17 18:02:17 2020][8592622.280812] .rt_nr_running : 1 [Tue Mar 17 18:02:17 2020][8592622.285607] .rt_throttled : 0 [Tue Mar 17 18:02:17 2020][8592622.290398] .rt_time : 0.000000 [Tue Mar 17 18:02:17 2020][8592622.295797] .rt_runtime : 950.000000 [Tue Mar 17 18:02:17 2020][8592622.301370] [Tue Mar 17 18:02:17 2020][8592622.303042] runnable tasks: [Tue Mar 17 18:02:17 2020][8592622.306016] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:17 2020][8592622.316960] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:17 2020][8592622.327831] watchdog/6 41 -8.960626 2148089 0 0.000000 24103.584343 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.339167] migration/6 42 0.000000 108995 0 0.000000 184562.531876 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.350501] ksoftirqd/6 43 206642737.066285 428315492 120 0.000000 5944261.910268 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.361837] kworker/6:0H 45 1369.249969 10 100 0.000000 0.048775 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.373217] kworker/6:1H 5604 206642737.066390 1227985737 100 0.000000 6741306.869714 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.384641] ccp-1-q2 6418 1574.140076 2 120 0.000000 0.012925 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.396041] kdmwork-253:56 18223 635219.142425 96171 120 0.000000 1007.823860 0.000000 2 / [Tue Mar 17 18:02:17 2020][8592622.407397] kdmwork-253:109 25207 635219.062574 63237 120 0.000000 743.814387 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.418740] kdmwork-253:117 27832 635219.248373 84765 120 0.000000 995.886285 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.430135] ll_ost_create02 68532 205155734.560689 6307670 120 0.000000 121812.767343 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.441531] ll_ost02_088 98679 205280516.957915 32816504 120 0.000000 987570.730007 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.452865] ll_ost02_093 98684 205280516.956431 48138725 120 0.000000 1374813.087837 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.464218] ldlm_cb02_003 103707 205956013.822177 2041 120 0.000000 34.753976 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.475654] ldlm_cn02_018 31071 206642554.627786 26396117 120 0.000000 622973.793299 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.486988] ldlm_cb02_005 52157 205956013.815955 1725 120 0.000000 31.816831 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.498323] ldlm_cn02_026 64119 206642548.233656 18610152 120 0.000000 431431.834528 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.509732] kworker/6:2 7183 206642737.066285 35653 120 0.000000 617.323200 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.521065] kworker/6:0 9754 206459902.962210 2 120 0.000000 0.012072 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.532407] R bash 14041 206642746.620350 68 120 0.000000 26.940039 0.000000 2 / [Tue Mar 17 18:02:18 2020][8592622.543772] [Tue Mar 17 18:02:18 2020][8592622.545446] cpu#7, 1996.233 MHz [Tue Mar 17 18:02:18 2020][8592622.548765] .nr_running : 0 [Tue Mar 17 18:02:18 2020][8592622.553558] .load : 0 [Tue Mar 17 18:02:18 2020][8592622.558352] .nr_switches : 13727370361 [Tue Mar 17 18:02:18 2020][8592622.564025] .nr_load_updates : 3297047076 [Tue Mar 17 18:02:18 2020][8592622.569601] .nr_uninterruptible : -161620 [Tue Mar 17 18:02:18 2020][8592622.574913] .next_balance : 12887.267075 [Tue Mar 17 18:02:18 2020][8592622.580673] .curr->pid : 0 [Tue Mar 17 18:02:18 2020][8592622.585469] .clock : 8592622585.352621 [Tue Mar 17 18:02:18 2020][8592622.591648] .cpu_load[0] : 0 [Tue Mar 17 18:02:18 2020][8592622.596456] .cpu_load[1] : 0 [Tue Mar 17 18:02:18 2020][8592622.601251] .cpu_load[2] : 0 [Tue Mar 17 18:02:18 2020][8592622.606061] .cpu_load[3] : 0 [Tue Mar 17 18:02:18 2020][8592622.610855] .cpu_load[4] : 0 [Tue Mar 17 18:02:18 2020][8592622.615661] .avg_idle : 8751970 [Tue Mar 17 18:02:18 2020][8592622.620978] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:18 2020][8592622.626290] [Tue Mar 17 18:02:18 2020][8592622.627961] cfs_rq[7]:/ [Tue Mar 17 18:02:18 2020][8592622.630585] .exec_clock : 0.000000 [Tue Mar 17 18:02:18 2020][8592622.635985] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:18 2020][8592622.641385] .min_vruntime : 181734965.745124 [Tue Mar 17 18:02:18 2020][8592622.647476] .max_vruntime : 0.000001 [Tue Mar 17 18:02:18 2020][8592622.652877] .spread : 0.000000 [Tue Mar 17 18:02:18 2020][8592622.658274] .spread0 : -135480170.321963 [Tue Mar 17 18:02:18 2020][8592622.664454] .nr_spread_over : 0 [Tue Mar 17 18:02:18 2020][8592622.669247] .nr_running : 0 [Tue Mar 17 18:02:18 2020][8592622.674040] .load : 0 [Tue Mar 17 18:02:18 2020][8592622.678833] .runnable_load_avg : 0 [Tue Mar 17 18:02:18 2020][8592622.683626] .blocked_load_avg : 0 [Tue Mar 17 18:02:18 2020][8592622.688418] .tg_load_avg : 0 [Tue Mar 17 18:02:18 2020][8592622.693210] .tg_load_contrib : 0 [Tue Mar 17 18:02:18 2020][8592622.698003] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:18 2020][8592622.702796] .tg->runnable_avg : 0 [Tue Mar 17 18:02:18 2020][8592622.707589] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:18 2020][8592622.712382] .throttled : 0 [Tue Mar 17 18:02:18 2020][8592622.717175] .throttle_count : 0 [Tue Mar 17 18:02:18 2020][8592622.721968] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:18 2020][8592622.726761] .avg->runnable_avg_period : 47919 [Tue Mar 17 18:02:18 2020][8592622.731902] [Tue Mar 17 18:02:18 2020][8592622.733571] rt_rq[7]:/ [Tue Mar 17 18:02:18 2020][8592622.736110] .rt_nr_running : 0 [Tue Mar 17 18:02:18 2020][8592622.740904] .rt_throttled : 0 [Tue Mar 17 18:02:18 2020][8592622.745695] .rt_time : 0.000000 [Tue Mar 17 18:02:18 2020][8592622.751096] .rt_runtime : 950.000000 [Tue Mar 17 18:02:18 2020][8592622.756668] [Tue Mar 17 18:02:18 2020][8592622.758339] runnable tasks: [Tue Mar 17 18:02:18 2020][8592622.761313] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:18 2020][8592622.772260] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:18 2020][8592622.783127] watchdog/7 46 -8.960817 2148152 0 0.000000 21551.324793 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.794465] migration/7 47 0.000000 60208 0 0.000000 78235.936223 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.805800] ksoftirqd/7 48 181734793.292068 549845757 120 0.000000 9432937.136079 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.817137] kworker/7:0H 50 1231.245567 10 100 0.000000 0.032690 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.828370] LustreError: dumping log to /tmp/lustre-log.1584493338.66920 [Tue Mar 17 18:02:18 2020][8592622.835384] kworker/7:1H 5603 181734793.304635 1034419347 100 0.000000 6407738.074357 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.846808] multipathd 17894 0.000000 4 0 0.000000 0.831536 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.858146] multipathd 17896 0.000000 5 0 0.000000 0.690832 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.869483] multipathd 17902 0.000000 5 0 0.000000 0.859849 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.880819] multipathd 17904 0.000000 5 0 0.000000 0.873736 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.892155] multipathd 17907 0.000000 4 0 0.000000 0.805979 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.903493] multipathd 17911 0.000000 4 0 0.000000 0.815156 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.914829] multipathd 17913 0.000000 7 0 0.000000 0.895760 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.926166] multipathd 17922 0.000000 4 0 0.000000 0.844660 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.937499] multipathd 17923 0.000000 6 0 0.000000 1.625923 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.948836] multipathd 17925 0.000000 6 0 0.000000 0.511734 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.960172] multipathd 17927 0.000000 5 0 0.000000 0.512174 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.971508] multipathd 17931 0.000000 5 0 0.000000 0.545959 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.982843] multipathd 17933 0.000000 4 0 0.000000 0.545498 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592622.994179] multipathd 17935 0.000000 5 0 0.000000 0.521612 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.005517] multipathd 17937 0.000000 6 0 0.000000 0.596524 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.016852] multipathd 17940 0.000000 4 0 0.000000 0.615570 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.028189] multipathd 17942 0.000000 5 0 0.000000 0.588689 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.039525] multipathd 17944 0.000000 4 0 0.000000 0.552701 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.050860] multipathd 17946 0.000000 4 0 0.000000 0.558732 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.062211] kdmflush 12448 3326.239350 2 100 0.000000 0.013465 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.073540] bioset 12454 3326.252603 2 100 0.000000 0.012374 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.084878] kdmflush 12456 3326.262440 2 100 0.000000 0.009858 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.096213] bioset 12457 3326.265364 2 100 0.000000 0.010079 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.107550] kdmflush 12473 3326.275102 2 100 0.000000 0.011021 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.118885] bioset 12474 3326.278265 2 100 0.000000 0.010690 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.130223] kdmflush 12482 3326.288887 2 100 0.000000 0.011812 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.141559] bioset 12483 3326.292242 2 100 0.000000 0.010911 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.152893] kdmflush 12485 3326.303840 2 100 0.000000 0.010680 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.164229] bioset 12486 3326.307582 2 100 0.000000 0.010550 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.175565] kdmflush 12512 3326.318483 2 100 0.000000 0.010951 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.186901] bioset 12513 3326.321699 2 100 0.000000 0.010379 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.198238] kdmflush 12515 3326.332467 2 100 0.000000 0.010992 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.209575] bioset 12516 3326.336222 2 100 0.000000 0.010871 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.220910] kdmflush 12521 3326.347133 2 100 0.000000 0.012523 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.232247] bioset 12522 3326.350819 2 100 0.000000 0.016091 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.243583] kdmflush 12566 3326.361752 2 100 0.000000 0.011802 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.254920] kdmflush 12589 3326.365070 2 100 0.000000 0.011591 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.266254] bioset 12590 3326.368304 2 100 0.000000 0.010459 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.277590] kdmflush 12597 3326.378888 2 100 0.000000 0.010690 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.288928] bioset 12598 3326.382174 2 100 0.000000 0.011602 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.300265] kdmflush 12654 3333.353606 2 100 0.000000 0.010600 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.311598] bioset 12655 3333.357232 2 100 0.000000 0.012745 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.322935] kdmflush 12657 3333.368090 2 100 0.000000 0.011391 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.334271] bioset 12658 3333.371912 2 100 0.000000 0.010770 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.345608] kdmflush 12663 3333.383186 2 100 0.000000 0.010549 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.356943] bioset 12664 3333.386788 2 100 0.000000 0.010339 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.368279] kdmflush 12666 3333.397827 2 100 0.000000 0.011462 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.379616] bioset 12667 3333.401419 2 100 0.000000 0.010110 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.390951] kdmflush 12669 3333.411662 2 100 0.000000 0.010059 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.402287] bioset 12670 3333.415284 2 100 0.000000 0.010409 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.413623] kdmflush 12674 3333.425484 2 100 0.000000 0.010940 0.000000 3 / [Tue Mar 17 18:02:18 2020][8592623.424961] bioset 12675 3333.429166 2 100 0.000000 0.010710 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.436295] kdmflush 12677 3333.439610 2 100 0.000000 0.010170 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.447633] bioset 12678 3333.442782 2 100 0.000000 0.010149 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.458969] kdmflush 12680 3333.453078 2 100 0.000000 0.010680 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.470303] bioset 12681 3333.456361 2 100 0.000000 0.010328 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.481640] kdmflush 12683 3333.466735 2 100 0.000000 0.010540 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.492976] bioset 12684 3333.470097 2 100 0.000000 0.010249 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.504311] kdmflush 12686 3333.480414 2 100 0.000000 0.010610 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.515648] bioset 12687 3333.484118 2 100 0.000000 0.010900 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.526984] kdmflush 12689 3333.494634 2 100 0.000000 0.010370 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.538321] bioset 12690 3333.498459 2 100 0.000000 0.010680 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.549659] bioset 12693 3333.508736 2 100 0.000000 0.010690 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.560993] kdmflush 12695 3333.519404 2 100 0.000000 0.010851 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.572330] bioset 12696 3333.522746 2 100 0.000000 0.010429 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.583665] kdmflush 12698 3333.533562 2 100 0.000000 0.010750 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.595000] bioset 12699 3333.536998 2 100 0.000000 0.010581 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.606338] kdmwork-253:27 12700 181734803.923552 877321844 120 0.000000 30602787.013047 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.617672] kdmflush 12701 3333.547673 2 100 0.000000 0.010069 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.629009] bioset 12702 3333.552297 2 100 0.000000 0.011552 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.640347] kdmflush 12704 3333.563192 2 100 0.000000 0.010439 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.651681] bioset 12705 3333.567036 2 100 0.000000 0.010961 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.663020] kdmflush 12707 3333.582502 2 100 0.000000 0.010419 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.674353] bioset 12708 3333.586263 2 100 0.000000 0.010330 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.685691] kdmflush 12710 3333.597543 2 100 0.000000 0.010851 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.697027] bioset 12711 3333.601277 2 100 0.000000 0.010700 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.708362] kdmflush 12713 3333.612014 2 100 0.000000 0.010129 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.719698] bioset 12714 3333.616487 2 100 0.000000 0.011543 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.731035] kdmflush 12716 3333.627204 2 100 0.000000 0.010741 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.742369] bioset 12717 3333.630626 2 100 0.000000 0.010539 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.753706] kdmflush 12719 3333.640982 2 100 0.000000 0.010309 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.765043] bioset 12720 3333.644364 2 100 0.000000 0.010480 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.776380] kdmflush 12722 3333.655179 2 100 0.000000 0.010599 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.787714] bioset 12723 3333.658713 2 100 0.000000 0.011141 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.799052] kdmflush 12725 3333.668787 2 100 0.000000 0.010740 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.810387] bioset 12726 3333.672410 2 100 0.000000 0.010490 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.821724] kdmflush 12729 3333.682630 2 100 0.000000 0.011021 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.833059] bioset 12730 3333.686383 2 100 0.000000 0.010920 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.844395] kdmflush 12732 3333.697052 2 100 0.000000 0.011351 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.855731] bioset 12733 3333.700706 2 100 0.000000 0.010730 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.867066] kdmflush 12754 3333.711194 2 100 0.000000 0.011542 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.878404] bioset 12755 3333.714606 2 100 0.000000 0.010479 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.889741] kdmflush 13412 3333.871976 2 100 0.000000 0.012203 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.901075] bioset 13413 3333.875500 2 100 0.000000 0.010990 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.912421] kdmflush 18150 3606.187722 2 100 0.000000 0.011291 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.923756] bioset 18151 3606.191197 2 100 0.000000 0.010770 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.935094] kdmflush 18168 3607.039036 2 100 0.000000 0.012104 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.946429] bioset 18169 3607.041983 2 100 0.000000 0.010470 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.957764] kdmflush 18179 3608.297598 2 100 0.000000 0.012272 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.969101] bioset 18180 3608.300502 2 100 0.000000 0.010440 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.980437] kdmflush 18189 3609.134138 2 100 0.000000 0.011151 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592623.991773] bioset 18191 3609.137541 2 100 0.000000 0.010318 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.003110] kdmflush 18221 3610.778580 2 100 0.000000 0.011652 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.014446] bioset 18222 3610.781524 2 100 0.000000 0.010400 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.025780] kdmflush 18231 3611.594413 2 100 0.000000 0.010790 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.037118] bioset 18232 3611.598006 2 100 0.000000 0.011191 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.048453] kdmflush 18241 3612.417485 2 100 0.000000 0.011601 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.059791] bioset 18242 3612.420338 2 100 0.000000 0.010049 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.071125] kdmflush 18266 3613.644246 2 100 0.000000 0.010701 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.082462] bioset 18267 3613.646869 2 100 0.000000 0.009688 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.093798] kdmflush 18275 3614.905259 2 100 0.000000 0.009217 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.105133] bioset 18276 3614.908181 2 100 0.000000 0.008446 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.116470] kdmflush 18283 3616.129004 2 100 0.000000 0.009568 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.127805] bioset 18284 3616.132107 2 100 0.000000 0.008897 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.139142] kdmwork-253:61 18285 485377.131494 74179 120 0.000000 905.124208 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.150477] kdmflush 18291 3618.163284 2 100 0.000000 0.009468 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.161815] bioset 18292 3618.166526 2 100 0.000000 0.009066 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.173150] kdmflush 18295 3618.983939 2 100 0.000000 0.009177 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.184485] bioset 18296 3618.987462 2 100 0.000000 0.008987 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.195822] kdmflush 18299 3619.793229 2 100 0.000000 0.008556 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.207158] bioset 18300 3619.796883 2 100 0.000000 0.011352 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.218494] kdmflush 18318 3621.464266 2 100 0.000000 0.011883 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.229830] bioset 18319 3621.467961 2 100 0.000000 0.010600 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.241166] kdmflush 18322 3622.304460 2 100 0.000000 0.011281 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.252502] bioset 18323 3622.307924 2 100 0.000000 0.010641 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.263839] kdmflush 18328 3623.113185 2 100 0.000000 0.010942 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.275175] bioset 18329 3623.116829 2 100 0.000000 0.010570 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.286511] kdmflush 18334 3623.949893 2 100 0.000000 0.011591 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.297847] bioset 18335 3623.953296 2 100 0.000000 0.010450 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.309183] kdmflush 18341 3624.774430 2 100 0.000000 0.012173 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.320518] bioset 18342 3624.778425 2 100 0.000000 0.011171 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.331855] kdmflush 18381 3626.078384 2 100 0.000000 0.010149 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.343193] bioset 18382 3626.081867 2 100 0.000000 0.008395 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.354528] kdmflush 18388 3626.924880 2 100 0.000000 0.009869 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.365863] bioset 18389 3626.928414 2 100 0.000000 0.008706 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.377201] kdmflush 18399 3628.207014 2 100 0.000000 0.011161 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.388534] bioset 18400 3628.210900 2 100 0.000000 0.012684 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.399872] kdmflush 18411 3629.100320 2 100 0.000000 0.012874 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.411208] bioset 18412 3629.104045 2 100 0.000000 0.012033 0.000000 3 / [Tue Mar 17 18:02:19 2020][8592624.422547] kdmflush 18427 3630.622898 2 100 0.000000 0.012004 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.433880] bioset 18441 3630.624703 2 100 0.000000 0.010159 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.445215] bioset 18488 3630.852010 2 100 0.000000 0.012313 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.456552] kdmflush 18504 3631.240656 2 100 0.000000 0.012894 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.467889] kdmflush 18515 3631.244113 2 100 0.000000 0.011832 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.479225] bioset 18516 3631.247417 2 100 0.000000 0.011572 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.490560] bioset 18524 3631.259088 2 100 0.000000 0.011711 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.501895] kdmwork-253:81 18525 485376.588628 87313 120 0.000000 1035.427402 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.513232] kdmflush 18529 3631.269197 2 100 0.000000 0.010810 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.524569] bioset 18532 3631.272371 2 100 0.000000 0.011441 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.535908] kdmflush 18678 3646.846296 2 100 0.000000 0.012703 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.547240] bioset 18680 3646.849490 2 100 0.000000 0.010079 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.558575] bioset 18692 3647.668124 2 100 0.000000 0.011261 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.569913] kdmwork-253:91 18694 485377.104448 62495 120 0.000000 696.485317 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.581247] kdmflush 18698 3647.679676 2 100 0.000000 0.012002 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.592584] kdmflush 18812 3648.978885 2 100 0.000000 0.018745 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.603922] kdmflush 19065 3677.945569 2 100 0.000000 0.011752 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.615257] kdmflush 19144 3678.285392 2 100 0.000000 0.011321 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.626594] bioset 19145 3678.288475 2 100 0.000000 0.011571 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.637928] kdmflush 19157 3678.298023 2 100 0.000000 0.012273 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.649264] bioset 19158 3678.301627 2 100 0.000000 0.011091 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.660602] kdmflush 19254 3678.534739 2 100 0.000000 0.011282 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.671936] bioset 19258 3678.537642 2 100 0.000000 0.010289 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.683272] kdmflush 19368 3680.251400 2 100 0.000000 0.012133 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.694610] kdmflush 19389 3680.264306 2 100 0.000000 0.011601 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.705946] bioset 19390 3680.267299 2 100 0.000000 0.010450 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.717281] kdmflush 19612 3680.279042 2 100 0.000000 0.009418 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.728616] bioset 19613 3680.281781 2 100 0.000000 0.008987 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.739956] kdmflush 19991 3710.632559 2 100 0.000000 0.010229 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.751291] bioset 19993 3710.636445 2 100 0.000000 0.011812 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.762626] kdmflush 20094 3712.237749 2 100 0.000000 0.009838 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.773962] bioset 20095 3712.241324 2 100 0.000000 0.011131 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.785298] kdmflush 20661 3725.901615 2 100 0.000000 0.013916 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.796639] kdmflush 25245 5346.142694 2 100 0.000000 0.014748 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.807973] bioset 26034 5852.443687 2 100 0.000000 0.012353 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.819310] kdmflush 31129 7570.732013 2 100 0.000000 0.011601 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.830646] irqbalance 50200 181734965.745124 1470182 120 0.000000 6618127.724286 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.842011] ll_ost03_000 66921 172653777.469818 20865001 120 0.000000 620635.495936 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.853354] ll_evictor 67051 181634898.001406 9852 120 0.000000 41520.774105 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.864697] ll_ost03_008 68346 181734914.637514 21267552 120 0.000000 610719.131921 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.876047] ll_ost_io03_015 68498 181734284.277902 81504809 120 0.000000 4841577.699484 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.887386] ll_ost_io03_018 68505 181734235.313630 91575255 120 0.000000 5738955.972024 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.898726] ll_ost03_023 68552 181734930.870076 15306968 120 0.000000 480611.045709 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.910072] ll_ost03_037 68609 181734890.665510 15163625 120 0.000000 453628.981528 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.921423] ll_ost_io03_023 68757 181734768.160575 152960974 120 0.000000 9061190.857104 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.932766] ll_ost_io03_035 68872 181734197.817511 133706789 120 0.000000 7916372.864586 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.944100] ll_ost_io03_040 68885 181734708.342971 74502590 120 0.000000 4334614.913685 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.955438] ldlm_bl_05 69232 181734953.752238 6334577 120 0.000000 87182.694552 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.966785] ldlm_cn03_008 74194 181734953.821569 8227052 120 0.000000 194158.142982 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.978116] ldlm_cn03_010 79032 181734862.279511 23049767 120 0.000000 516467.601253 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592624.989465] ldlm_bl_39 100120 181734946.513973 6105034 120 0.000000 83683.557054 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.000893] bash 102158 496214.861471 15 120 0.000000 6.517175 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.012334] ldlm_cn03_022 64128 181732906.020690 4283851 120 0.000000 90748.669008 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.023667] ldlm_cn03_024 64130 181316546.058145 29601865 120 0.000000 684611.449124 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.035005] ldlm_cn03_025 64131 181732906.015250 18345429 120 0.000000 412135.932255 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.046355] ll_ost_io03_074 90873 181734780.368300 174806426 120 0.000000 10523865.194366 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.057709] ll_ost_io03_083 40796 181734207.140080 46468910 120 0.000000 2660462.036150 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.069047] ll_ost_io03_086 40805 181734189.725943 45625754 120 0.000000 2762550.320438 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.080385] ll_ost_io03_091 40824 181734602.696947 82448513 120 0.000000 4673687.225432 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.091721] ll_ost_io03_092 40829 181734260.844262 133399910 120 0.000000 7649753.241114 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.103062] ll_ost_io03_109 40862 181734490.455697 59642064 120 0.000000 3243127.072146 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.114408] ll_ost03_075 70321 181734930.800676 13323511 120 0.000000 363002.075374 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.125746] ll_ost03_078 70325 181734965.809937 6529550 120 0.000000 190925.392760 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.137081] ll_ost03_084 70523 181734793.437640 11821656 120 0.000000 355675.200053 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.148417] ll_ost03_087 71193 172653777.470549 11016667 120 0.000000 320370.493664 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.159754] ll_ost03_092 74373 179021225.148360 17077866 120 0.000000 510771.843129 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.171089] ll_ost03_096 74617 179021225.151716 7561432 120 0.000000 261730.883212 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.182444] kworker/7:0 73766 181682546.125390 68 120 0.000000 0.203704 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.193790] kworker/7:2 13142 181734953.751847 4368 120 0.000000 110.024693 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.205124] kworker/u260:0 13411 181734946.565843 1695 120 0.000000 169.515666 0.000000 3 / [Tue Mar 17 18:02:20 2020][8592625.216521] [Tue Mar 17 18:02:20 2020][8592625.218194] cpu#8, 1996.233 MHz [Tue Mar 17 18:02:20 2020][8592625.221514] .nr_running : 1 [Tue Mar 17 18:02:20 2020][8592625.226307] .load : 1024 [Tue Mar 17 18:02:20 2020][8592625.231365] .nr_switches : 26246580393 [Tue Mar 17 18:02:20 2020][8592625.237044] .nr_load_updates : 4153216169 [Tue Mar 17 18:02:20 2020][8592625.242618] .nr_uninterruptible : 45346 [Tue Mar 17 18:02:20 2020][8592625.247759] .next_balance : 12887.269755 [Tue Mar 17 18:02:20 2020][8592625.253532] .curr->pid : 11793 [Tue Mar 17 18:02:20 2020][8592625.258671] .clock : 8592625258.363367 [Tue Mar 17 18:02:20 2020][8592625.264849] .cpu_load[0] : 1024 [Tue Mar 17 18:02:20 2020][8592625.269918] .cpu_load[1] : 1024 [Tue Mar 17 18:02:20 2020][8592625.274971] .cpu_load[2] : 1024 [Tue Mar 17 18:02:20 2020][8592625.280026] .cpu_load[3] : 1024 [Tue Mar 17 18:02:20 2020][8592625.285091] .cpu_load[4] : 1024 [Tue Mar 17 18:02:20 2020][8592625.290147] .avg_idle : 9151608 [Tue Mar 17 18:02:20 2020][8592625.295474] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:20 2020][8592625.300790] [Tue Mar 17 18:02:20 2020][8592625.302460] cfs_rq[8]:/ [Tue Mar 17 18:02:20 2020][8592625.305086] .exec_clock : 0.000000 [Tue Mar 17 18:02:20 2020][8592625.310485] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:20 2020][8592625.315882] .min_vruntime : 301981193.164256 [Tue Mar 17 18:02:20 2020][8592625.321978] .max_vruntime : 0.000001 [Tue Mar 17 18:02:20 2020][8592625.327376] .spread : 0.000000 [Tue Mar 17 18:02:20 2020][8592625.332776] .spread0 : -15233945.829324 [Tue Mar 17 18:02:20 2020][8592625.338868] .nr_spread_over : 0 [Tue Mar 17 18:02:20 2020][8592625.343660] .nr_running : 1 [Tue Mar 17 18:02:20 2020][8592625.348454] .load : 1024 [Tue Mar 17 18:02:20 2020][8592625.353506] .runnable_load_avg : 1023 [Tue Mar 17 18:02:20 2020][8592625.358558] .blocked_load_avg : 0 [Tue Mar 17 18:02:20 2020][8592625.363350] .tg_load_avg : 0 [Tue Mar 17 18:02:20 2020][8592625.368144] .tg_load_contrib : 0 [Tue Mar 17 18:02:20 2020][8592625.372938] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:20 2020][8592625.377729] .tg->runnable_avg : 0 [Tue Mar 17 18:02:20 2020][8592625.382522] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:20 2020][8592625.387314] .throttled : 0 [Tue Mar 17 18:02:20 2020][8592625.392108] .throttle_count : 0 [Tue Mar 17 18:02:20 2020][8592625.396901] .avg->runnable_avg_sum : 46860 [Tue Mar 17 18:02:20 2020][8592625.402038] .avg->runnable_avg_period : 46704 [Tue Mar 17 18:02:20 2020][8592625.407182] [Tue Mar 17 18:02:20 2020][8592625.408853] rt_rq[8]:/ [Tue Mar 17 18:02:20 2020][8592625.411391] .rt_nr_running : 0 [Tue Mar 17 18:02:20 2020][8592625.416184] .rt_throttled : 0 [Tue Mar 17 18:02:21 2020][8592625.420977] .rt_time : 0.000000 [Tue Mar 17 18:02:21 2020][8592625.426377] .rt_runtime : 950.000000 [Tue Mar 17 18:02:21 2020][8592625.431948] [Tue Mar 17 18:02:21 2020][8592625.433622] runnable tasks: [Tue Mar 17 18:02:21 2020][8592625.436594] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:21 2020][8592625.447541] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:21 2020][8592625.458409] watchdog/8 51 -11.978359 2148153 0 0.000000 19966.191036 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.469744] migration/8 52 0.000000 99756 0 0.000000 93254.378480 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.481081] ksoftirqd/8 53 301719110.491680 625621298 120 0.000000 12672338.979517 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.492416] kworker/8:0H 55 1090.584364 10 100 0.000000 0.036319 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.503786] writeback 258 262099977.802391 8 100 0.000000 4.948100 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.515133] kworker/8:1H 4771 301721354.979961 1217938595 100 0.000000 8421156.517502 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.526678] ldlm_cn00_002 66885 301718879.382791 15213815 120 0.000000 330066.164616 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.538015] ll_ost00_002 66907 291203334.040016 25457609 120 0.000000 529617.885268 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.549390] ll_ost00_024 68585 291203333.926061 51330399 120 0.000000 1091415.260415 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.560737] ll_ost00_045 68672 291203333.921674 49340622 120 0.000000 1072348.345230 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.572075] ll_ost00_048 68685 291203333.992445 62190269 120 0.000000 1307782.176741 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.583414] ll_ost00_059 68738 291203336.397806 40492105 120 0.000000 861566.536247 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.594753] ll_ost00_067 68805 291203334.110389 32078143 120 0.000000 731276.370023 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.606089] ll_ost00_069 68807 291203333.960517 45519543 120 0.000000 964836.779283 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.617427] ll_ost00_078 68825 296024163.762743 62992896 120 0.000000 1376890.892663 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.628805] ldlm_cn00_010 111306 301718872.602864 28788608 120 0.000000 654450.989109 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.640235] ldlm_cn00_025 30589 301718884.327485 18704602 120 0.000000 446981.133147 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.651620] ll_ost00_082 101323 296262149.822400 33897887 120 0.000000 724402.460557 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.663052] ll_ost00_100 85269 291203333.943054 35383389 120 0.000000 785900.825275 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.674383] ll_ost00_102 85271 291203334.106252 15118301 120 0.000000 331029.366475 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.685733] kworker/8:1 38456 300340540.512442 103746 120 0.000000 2169.409010 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.697069] kworker/8:2 4643 301978763.174963 39387 120 0.000000 609.455890 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.708404] R lnetctl 11793 301981592.164492 6357 120 0.000000 3519833.064322 0.000000 0 / [Tue Mar 17 18:02:21 2020][8592625.719780] [Tue Mar 17 18:02:21 2020][8592625.721454] cpu#9, 1996.233 MHz [Tue Mar 17 18:02:21 2020][8592625.724775] .nr_running : 0 [Tue Mar 17 18:02:21 2020][8592625.729567] .load : 0 [Tue Mar 17 18:02:21 2020][8592625.734362] .nr_switches : 12058086289 [Tue Mar 17 18:02:21 2020][8592625.740044] .nr_load_updates : 2966295953 [Tue Mar 17 18:02:21 2020][8592625.745632] .nr_uninterruptible : -186651 [Tue Mar 17 18:02:21 2020][8592625.750948] .next_balance : 12887.270252 [Tue Mar 17 18:02:21 2020][8592625.756708] .curr->pid : 0 [Tue Mar 17 18:02:21 2020][8592625.761504] .clock : 8592625761.366984 [Tue Mar 17 18:02:21 2020][8592625.767684] .cpu_load[0] : 0 [Tue Mar 17 18:02:21 2020][8592625.772491] .cpu_load[1] : 0 [Tue Mar 17 18:02:21 2020][8592625.777287] .cpu_load[2] : 0 [Tue Mar 17 18:02:21 2020][8592625.782081] .cpu_load[3] : 0 [Tue Mar 17 18:02:21 2020][8592625.786872] .cpu_load[4] : 0 [Tue Mar 17 18:02:21 2020][8592625.791680] .avg_idle : 10000000 [Tue Mar 17 18:02:21 2020][8592625.797097] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:21 2020][8592625.802411] [Tue Mar 17 18:02:21 2020][8592625.804081] cfs_rq[9]:/ [Tue Mar 17 18:02:21 2020][8592625.806707] .exec_clock : 0.000000 [Tue Mar 17 18:02:21 2020][8592625.812105] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:21 2020][8592625.817505] .min_vruntime : 179344821.174370 [Tue Mar 17 18:02:21 2020][8592625.823597] .max_vruntime : 0.000001 [Tue Mar 17 18:02:21 2020][8592625.828997] .spread : 0.000000 [Tue Mar 17 18:02:21 2020][8592625.834397] .spread0 : -137870318.526320 [Tue Mar 17 18:02:21 2020][8592625.840576] .nr_spread_over : 0 [Tue Mar 17 18:02:21 2020][8592625.845368] .nr_running : 0 [Tue Mar 17 18:02:21 2020][8592625.850160] .load : 0 [Tue Mar 17 18:02:21 2020][8592625.854955] .runnable_load_avg : 0 [Tue Mar 17 18:02:21 2020][8592625.859746] .blocked_load_avg : 0 [Tue Mar 17 18:02:21 2020][8592625.864539] .tg_load_avg : 0 [Tue Mar 17 18:02:21 2020][8592625.869332] .tg_load_contrib : 0 [Tue Mar 17 18:02:21 2020][8592625.874124] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:21 2020][8592625.878918] .tg->runnable_avg : 0 [Tue Mar 17 18:02:21 2020][8592625.883711] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:21 2020][8592625.888504] .throttled : 0 [Tue Mar 17 18:02:21 2020][8592625.893297] .throttle_count : 0 [Tue Mar 17 18:02:21 2020][8592625.898088] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:21 2020][8592625.902881] .avg->runnable_avg_period : 47779 [Tue Mar 17 18:02:21 2020][8592625.908023] [Tue Mar 17 18:02:21 2020][8592625.909693] rt_rq[9]:/ [Tue Mar 17 18:02:21 2020][8592625.912232] .rt_nr_running : 0 [Tue Mar 17 18:02:21 2020][8592625.917026] .rt_throttled : 0 [Tue Mar 17 18:02:21 2020][8592625.921817] .rt_time : 0.000000 [Tue Mar 17 18:02:21 2020][8592625.927217] .rt_runtime : 950.000000 [Tue Mar 17 18:02:21 2020][8592625.932791] [Tue Mar 17 18:02:21 2020][8592625.934463] runnable tasks: [Tue Mar 17 18:02:21 2020][8592625.937436] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:21 2020][8592625.948381] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:21 2020][8592625.959250] watchdog/9 56 -11.979321 2148154 0 0.000000 20063.472783 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592625.970586] migration/9 57 0.000000 75642 0 0.000000 76151.181590 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592625.981922] ksoftirqd/9 58 179344735.933070 468592663 120 0.000000 9049741.822237 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592625.993257] kworker/9:0H 60 3304.980838 10 100 0.000000 0.045275 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.004627] khungtaskd 257 179344809.593510 72355 120 0.000000 26806.848857 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.015993] multipathd 17949 0.000000 10904 0 0.000000 1831.351060 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.027345] kdmwork-253:8 12569 179344766.007560 870599363 120 0.000000 31642163.933925 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.038701] kworker/9:1H 14306 179344765.994345 941407926 100 0.000000 6382558.401683 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.050051] kdmwork-253:63 18297 1004085.520956 84033 120 0.000000 988.587762 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.061397] kdmwork-253:87 18586 179344777.994253 804282180 120 0.000000 29271741.471460 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.072733] kdmwork-253:89 18664 179344766.005667 826388723 120 0.000000 29619720.930024 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.084092] cfs_rh_01 64740 179214948.115910 10349 120 0.000000 4274.522767 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.095443] ptlrpcd_01_08 64825 179344801.630693 4382825 120 0.000000 137734.836704 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.106779] md10_raid6 65692 179343924.728354 1414461193 120 0.000000 442914941.944463 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.118198] md4_raid6 65845 179343930.600971 1383638854 120 0.000000 428996585.882791 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.129624] kmmpd-md10 66556 179343924.676907 18355441 120 0.000000 148689.672868 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.140980] ll_ost_io01_004 68355 179344632.736026 128080826 120 0.000000 8217497.898874 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.152324] ll_ost_io01_010 68385 179343961.070677 79412527 120 0.000000 5763340.936753 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.163677] ll_ost01_036 68612 179344227.399508 21023540 120 0.000000 559857.341569 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.175013] ll_ost01_046 68653 179344777.378894 20067247 120 0.000000 523612.226617 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.186345] ll_ost01_047 68660 179344080.234327 21622831 120 0.000000 594556.872565 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.197683] ll_ost01_053 68679 173491268.648322 27875531 120 0.000000 723288.275791 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.209029] ll_ost01_075 68809 179344652.277742 22103363 120 0.000000 602306.773061 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.220363] ll_ost01_078 68824 179344086.844739 27890637 120 0.000000 742447.230814 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.231704] ll_ost_io01_049 68881 179344632.555726 90873220 120 0.000000 6335925.534354 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.243051] ll_ost_io01_060 71826 179343979.179109 42651123 120 0.000000 2846149.752881 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.254390] ll_ost_io01_071 73702 179344695.074038 72237765 120 0.000000 4840672.865124 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.265724] ll_ost_io01_079 73710 179344223.285678 48531309 120 0.000000 3758756.423633 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.277061] ll_ost_io01_089 73720 179343973.725309 65999536 120 0.000000 5341407.868503 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.288410] ldlm_bl_33 100114 179344809.191613 6110215 120 0.000000 83837.759000 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.299847] ldlm_bl_81 100162 179344809.196763 6113879 120 0.000000 83872.466545 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.311272] ldlm_cn01_019 111311 179344797.772096 12046211 120 0.000000 270335.670205 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.322694] ldlm_cn01_030 111460 179344809.621292 12807060 120 0.000000 279571.890619 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.334152] ll_ost_io01_111 40831 179343951.198210 51686943 120 0.000000 3178456.563791 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.345489] ll_ost01_094 101304 173491268.673329 21608988 120 0.000000 556518.182521 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.356915] ll_ost01_099 101309 173491269.755348 21035227 120 0.000000 550897.230433 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.368336] ll_ost01_102 101312 173491268.657490 18512037 120 0.000000 696981.409626 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.379795] kworker/9:2 4522 179344809.181293 57413 120 0.000000 903.782361 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.391141] kworker/9:1 13754 179324158.060132 2 120 0.000000 0.009328 0.000000 1 / [Tue Mar 17 18:02:21 2020][8592626.402501] [Tue Mar 17 18:02:21 2020][8592626.404176] cpu#10, 1996.233 MHz [Tue Mar 17 18:02:21 2020][8592626.407585] .nr_running : 0 [Tue Mar 17 18:02:21 2020][8592626.412391] .load : 0 [Tue Mar 17 18:02:21 2020][8592626.417185] .nr_switches : 18487694484 [Tue Mar 17 18:02:21 2020][8592626.422848] .nr_load_updates : 3832220577 [Tue Mar 17 18:02:22 2020][8592626.428442] .nr_uninterruptible : 15276050 [Tue Mar 17 18:02:22 2020][8592626.433844] .next_balance : 12887.270936 [Tue Mar 17 18:02:22 2020][8592626.439590] .curr->pid : 0 [Tue Mar 17 18:02:22 2020][8592626.444397] .clock : 8592626444.366864 [Tue Mar 17 18:02:22 2020][8592626.450580] .cpu_load[0] : 0 [Tue Mar 17 18:02:22 2020][8592626.455373] .cpu_load[1] : 0 [Tue Mar 17 18:02:22 2020][8592626.460172] .cpu_load[2] : 0 [Tue Mar 17 18:02:22 2020][8592626.464982] .cpu_load[3] : 0 [Tue Mar 17 18:02:22 2020][8592626.469777] .cpu_load[4] : 0 [Tue Mar 17 18:02:22 2020][8592626.474570] .avg_idle : 4231063 [Tue Mar 17 18:02:22 2020][8592626.479895] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:22 2020][8592626.485212] [Tue Mar 17 18:02:22 2020][8592626.486880] cfs_rq[10]:/ [Tue Mar 17 18:02:22 2020][8592626.489594] .exec_clock : 0.000000 [Tue Mar 17 18:02:22 2020][8592626.494992] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:22 2020][8592626.500391] .min_vruntime : 207337093.294895 [Tue Mar 17 18:02:22 2020][8592626.506484] .max_vruntime : 0.000001 [Tue Mar 17 18:02:22 2020][8592626.511883] .spread : 0.000000 [Tue Mar 17 18:02:22 2020][8592626.517282] .spread0 : -109878046.998793 [Tue Mar 17 18:02:22 2020][8592626.523463] .nr_spread_over : 0 [Tue Mar 17 18:02:22 2020][8592626.528255] .nr_running : 0 [Tue Mar 17 18:02:22 2020][8592626.533048] .load : 0 [Tue Mar 17 18:02:22 2020][8592626.537840] .runnable_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592626.542634] .blocked_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592626.547425] .tg_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592626.552219] .tg_load_contrib : 0 [Tue Mar 17 18:02:22 2020][8592626.557013] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:22 2020][8592626.561803] .tg->runnable_avg : 0 [Tue Mar 17 18:02:22 2020][8592626.566597] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:22 2020][8592626.571388] .throttled : 0 [Tue Mar 17 18:02:22 2020][8592626.576182] .throttle_count : 0 [Tue Mar 17 18:02:22 2020][8592626.580974] .avg->runnable_avg_sum : 31 [Tue Mar 17 18:02:22 2020][8592626.585855] .avg->runnable_avg_period : 46341 [Tue Mar 17 18:02:22 2020][8592626.590997] [Tue Mar 17 18:02:22 2020][8592626.592667] rt_rq[10]:/ [Tue Mar 17 18:02:22 2020][8592626.595292] .rt_nr_running : 0 [Tue Mar 17 18:02:22 2020][8592626.600084] .rt_throttled : 0 [Tue Mar 17 18:02:22 2020][8592626.604879] .rt_time : 0.000000 [Tue Mar 17 18:02:22 2020][8592626.610276] .rt_runtime : 950.000000 [Tue Mar 17 18:02:22 2020][8592626.615850] [Tue Mar 17 18:02:22 2020][8592626.617521] runnable tasks: [Tue Mar 17 18:02:22 2020][8592626.620494] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:22 2020][8592626.631442] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:22 2020][8592626.642313] watchdog/10 61 -11.979271 2148156 0 0.000000 21779.798743 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.653646] migration/10 62 0.000000 116511 0 0.000000 191383.176568 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.664980] ksoftirqd/10 63 207336970.577915 457614555 120 0.000000 6458805.091907 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.676317] kworker/10:0H 65 102897531.485644 12 100 0.000000 0.053008 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.687686] kswapd2 278 207335864.991935 4004374 120 0.000000 13767160.355613 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.699037] jbd2/sda2-8 5656 207337078.582265 2306409 120 0.000000 61721.391207 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.710412] kdmwork-253:38 12734 656919.154437 229573 120 0.000000 3165.687464 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.721786] kdmwork-253:105 20096 207336696.946204 810882285 120 0.000000 30602126.813270 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.733135] rs:main Q:Reg 50893 207337086.230939 958826 120 0.000000 43709.667664 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.744483] kiblnd_sd_02_02 64792 207337081.776667 11162282875 120 0.000000 88032416.610190 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.756000] ptlrpcd_02_07 64836 207335757.899352 2555420 120 0.000000 99263.893197 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.767335] ptlrpcd_02_08 64837 207335757.864086 2453696 120 0.000000 101138.911303 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.778687] ll_ost_io02_002 66968 207335788.945957 125289180 120 0.000000 8290688.707959 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.790027] ll_ost_seq02_00 66981 23826.733616 3 120 0.000000 0.073978 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.801397] ll_ost02_035 68599 207336870.348326 43222278 120 0.000000 1253445.459687 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.812733] ll_ost02_038 68613 206006409.490178 54872920 120 0.000000 1588033.652167 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.824068] ll_ost02_039 68620 207337082.368904 46011785 120 0.000000 1321125.660976 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.835412] ll_ost02_052 68683 206006409.490207 20671381 120 0.000000 600070.639697 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.846748] ll_ost02_053 68684 207337078.611719 40799075 120 0.000000 1142940.287053 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.858092] ll_ost_io02_024 68747 207336438.495366 115070051 120 0.000000 7888145.110429 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.869431] ll_ost_io02_027 68766 207336463.811959 141741569 120 0.000000 9743502.004838 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.880765] ll_ost02_069 68778 207337081.325942 35523096 120 0.000000 996355.132939 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.892103] ll_ost_io02_031 68788 207336336.395807 65392208 120 0.000000 5056702.661929 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.903439] ll_ost02_073 68792 207337039.569068 38212852 120 0.000000 1082343.596551 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.914773] ll_ost02_074 68793 203423437.961500 34272139 120 0.000000 997187.352177 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.926114] ll_ost_io02_036 68835 207336506.937473 102494062 120 0.000000 6589672.968473 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.937452] ll_ost_io02_046 68860 207336498.704329 139527214 120 0.000000 8547406.952871 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.948788] ldlm_cn02_004 68908 207335103.490429 18118266 120 0.000000 430088.505359 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.960137] ll_ost_create02 98628 205891267.118111 4456165 120 0.000000 85875.902339 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.971478] ldlm_bl_16 100097 207337078.534865 6106526 120 0.000000 83588.914834 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.982903] ldlm_bl_19 100100 207337074.272830 6105089 120 0.000000 83610.900584 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592626.994335] ldlm_bl_69 100150 207337078.524856 6110932 120 0.000000 83700.225401 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.005762] ldlm_bl_85 100166 207337078.530237 6106884 120 0.000000 83360.574177 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.017181] ldlm_bl_88 100169 207337081.304122 6111611 120 0.000000 83895.627405 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.028630] ll_ost_io02_068 90837 207335710.880982 78384625 120 0.000000 4581211.425336 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.039964] ll_ost_io02_073 90842 207335778.411192 105129225 120 0.000000 6719912.004618 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.051306] ll_ost_io02_083 90862 207336567.716958 78124287 120 0.000000 5788693.702058 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.062653] ll_ost_io02_100 40795 207336401.669915 154700885 120 0.000000 9683395.570506 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.073992] ll_ost_io02_105 40809 207335815.985267 86363776 120 0.000000 5003329.142532 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.085358] kworker/10:2H 116594 207337063.515287 650719356 100 0.000000 3845153.369568 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.096791] md2_resync 123020 207336338.012785 112576798 125 0.000000 4099085.759551 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.108221] kworker/10:1 3894 207337081.307018 69844 120 0.000000 883.568691 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.119562] kworker/10:2 13676 207312414.339061 2 120 0.000000 0.009509 0.000000 2 / [Tue Mar 17 18:02:22 2020][8592627.130937] [Tue Mar 17 18:02:22 2020][8592627.132615] cpu#11, 1996.233 MHz [Tue Mar 17 18:02:22 2020][8592627.136023] .nr_running : 0 [Tue Mar 17 18:02:22 2020][8592627.140814] .load : 0 [Tue Mar 17 18:02:22 2020][8592627.145607] .nr_switches : 13998441386 [Tue Mar 17 18:02:22 2020][8592627.151280] .nr_load_updates : 3306487109 [Tue Mar 17 18:02:22 2020][8592627.156873] .nr_uninterruptible : -167424 [Tue Mar 17 18:02:22 2020][8592627.162188] .next_balance : 12887.271664 [Tue Mar 17 18:02:22 2020][8592627.167947] .curr->pid : 0 [Tue Mar 17 18:02:22 2020][8592627.172744] .clock : 8592627172.367323 [Tue Mar 17 18:02:22 2020][8592627.178923] .cpu_load[0] : 0 [Tue Mar 17 18:02:22 2020][8592627.183729] .cpu_load[1] : 0 [Tue Mar 17 18:02:22 2020][8592627.188525] .cpu_load[2] : 0 [Tue Mar 17 18:02:22 2020][8592627.193317] .cpu_load[3] : 0 [Tue Mar 17 18:02:22 2020][8592627.198111] .cpu_load[4] : 0 [Tue Mar 17 18:02:22 2020][8592627.202918] .avg_idle : 8804884 [Tue Mar 17 18:02:22 2020][8592627.208248] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:22 2020][8592627.213563] [Tue Mar 17 18:02:22 2020][8592627.215232] cfs_rq[11]:/ [Tue Mar 17 18:02:22 2020][8592627.217944] .exec_clock : 0.000000 [Tue Mar 17 18:02:22 2020][8592627.223344] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:22 2020][8592627.228744] .min_vruntime : 186021858.849813 [Tue Mar 17 18:02:22 2020][8592627.234837] .max_vruntime : 0.000001 [Tue Mar 17 18:02:22 2020][8592627.240236] .spread : 0.000000 [Tue Mar 17 18:02:22 2020][8592627.245635] .spread0 : -131193282.426390 [Tue Mar 17 18:02:22 2020][8592627.251814] .nr_spread_over : 0 [Tue Mar 17 18:02:22 2020][8592627.256607] .nr_running : 0 [Tue Mar 17 18:02:22 2020][8592627.261399] .load : 0 [Tue Mar 17 18:02:22 2020][8592627.266191] .runnable_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592627.270986] .blocked_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592627.275779] .tg_load_avg : 0 [Tue Mar 17 18:02:22 2020][8592627.280571] .tg_load_contrib : 0 [Tue Mar 17 18:02:22 2020][8592627.285363] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:22 2020][8592627.290156] .tg->runnable_avg : 0 [Tue Mar 17 18:02:22 2020][8592627.294948] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:22 2020][8592627.299742] .throttled : 0 [Tue Mar 17 18:02:22 2020][8592627.304533] .throttle_count : 0 [Tue Mar 17 18:02:22 2020][8592627.309328] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:22 2020][8592627.314120] .avg->runnable_avg_period : 46312 [Tue Mar 17 18:02:22 2020][8592627.319262] [Tue Mar 17 18:02:22 2020][8592627.320931] rt_rq[11]:/ [Tue Mar 17 18:02:22 2020][8592627.323557] .rt_nr_running : 0 [Tue Mar 17 18:02:22 2020][8592627.328350] .rt_throttled : 0 [Tue Mar 17 18:02:22 2020][8592627.333142] .rt_time : 0.005881 [Tue Mar 17 18:02:22 2020][8592627.338542] .rt_runtime : 950.000000 [Tue Mar 17 18:02:22 2020][8592627.344115] [Tue Mar 17 18:02:22 2020][8592627.345787] runnable tasks: [Tue Mar 17 18:02:22 2020][8592627.348760] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:22 2020][8592627.359706] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:22 2020][8592627.370579] watchdog/11 66 -11.979141 2148155 0 0.000000 18630.714360 0.000000 3 / [Tue Mar 17 18:02:22 2020][8592627.381911] migration/11 67 0.000000 64504 0 0.000000 78519.797525 0.000000 3 / [Tue Mar 17 18:02:22 2020][8592627.393248] ksoftirqd/11 68 186021632.053026 547347832 120 0.000000 9377686.493993 0.000000 3 / [Tue Mar 17 18:02:22 2020][8592627.404582] kworker/11:0H 70 1185.852447 10 100 0.000000 0.033715 0.000000 3 / [Tue Mar 17 18:02:22 2020][8592627.415951] kworker/11:1H 5602 186021640.488938 1027354004 100 0.000000 6431482.913494 0.000000 3 / [Tue Mar 17 18:02:22 2020][8592627.427433] kdmwork-253:57 18233 521334.217817 226367 120 0.000000 3153.805018 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.438773] kdmwork-253:78 18494 186021638.200008 831782446 120 0.000000 23863547.036523 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.450119] kdmwork-253:98 19259 186021648.949050 810898678 120 0.000000 30411586.571681 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.461474] cfs_rh_02 64741 185887181.892670 10427 120 0.000000 4495.063508 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.472806] cfs_rh_03 64742 185887176.840739 10491 120 0.000000 4467.521839 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.484152] kiblnd_sd_03_03 64797 186021858.945674 9109691012 120 0.000000 69112005.147728 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.495575] lnet_discovery 64803 186021846.715972 8599185 120 0.000000 48520.563441 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.506918] ptlrpcd_03_00 64841 186021048.176977 3438229 120 0.000000 119199.973047 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.518257] md6_raid6 65600 186020979.282193 1411749471 120 0.000000 446940468.346338 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.529700] ll_ost_create03 68343 184533437.574876 3010586 120 0.000000 49740.888684 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.541034] ll_ost03_010 68356 186021658.528649 21737201 120 0.000000 657391.617677 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.552369] ll_ost03_012 68366 186021676.050096 16909572 120 0.000000 506441.315667 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.563707] ll_ost_io03_009 68394 186021082.685116 58873019 120 0.000000 4016567.879424 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.575052] ll_ost03_018 68543 186021652.209388 21157853 120 0.000000 635780.185261 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.586384] ll_ost03_019 68544 183099175.734338 15836080 120 0.000000 466875.239148 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.597721] ll_ost03_022 68551 176474267.370849 19659098 120 0.000000 578383.792468 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.609068] ll_ost03_046 68641 186021664.832455 24384064 120 0.000000 698714.711598 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.620402] ll_ost03_049 68647 186021668.848954 18626211 120 0.000000 547719.074947 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.631736] ll_ost03_050 68652 186021780.594207 20288383 120 0.000000 605849.723795 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.643090] ll_ost_io03_031 68829 186021586.230154 55480536 120 0.000000 3248708.051153 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.654434] ll_ost_io03_038 68879 186021638.321274 89055587 120 0.000000 5549219.482066 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.665774] ll_ost_io03_043 68899 186021646.034980 132365890 120 0.000000 8091127.721549 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.677134] ldlm_bl_18 100099 186021845.532366 6106921 120 0.000000 83716.827116 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.688569] ldlm_bl_84 100165 186021845.544999 6105465 120 0.000000 83865.871098 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.699988] ldlm_bl_87 100168 186021845.471461 6102895 120 0.000000 83401.430508 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.711458] ll_ost_io03_099 40845 186021634.431667 64207982 120 0.000000 3916408.221772 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.722804] ll_ost03_073 70319 186021648.287814 14735752 120 0.000000 387299.394792 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.734137] ll_ost03_080 70327 176474267.372102 14254969 120 0.000000 409171.553691 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.745472] ll_ost03_083 70330 186021843.467701 4665502 120 0.000000 137429.387006 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.756812] ll_ost03_098 78398 183099175.734628 3469503 120 0.000000 119843.562877 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.768150] ll_ost03_102 112801 186021648.691882 17671127 120 0.000000 463781.640304 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.779583] kworker/11:0 94215 186021843.683800 1423750 120 0.000000 21548.609433 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.790926] kworker/11:1 103067 184899121.934020 5 120 0.000000 0.053153 0.000000 3 / [Tue Mar 17 18:02:23 2020][8592627.802404] [Tue Mar 17 18:02:23 2020][8592627.804079] cpu#12, 1996.233 MHz [Tue Mar 17 18:02:23 2020][8592627.807487] .nr_running : 0 [Tue Mar 17 18:02:23 2020][8592627.812281] .load : 0 [Tue Mar 17 18:02:23 2020][8592627.817080] .nr_switches : 18619781554 [Tue Mar 17 18:02:23 2020][8592627.822753] .nr_load_updates : 3590901041 [Tue Mar 17 18:02:23 2020][8592627.828329] .nr_uninterruptible : -629434 [Tue Mar 17 18:02:23 2020][8592627.833642] .next_balance : 12887.272335 [Tue Mar 17 18:02:23 2020][8592627.839402] .curr->pid : 0 [Tue Mar 17 18:02:23 2020][8592627.844213] .clock : 8592627843.368504 [Tue Mar 17 18:02:23 2020][8592627.850396] .cpu_load[0] : 0 [Tue Mar 17 18:02:23 2020][8592627.855208] .cpu_load[1] : 0 [Tue Mar 17 18:02:23 2020][8592627.860007] .cpu_load[2] : 0 [Tue Mar 17 18:02:23 2020][8592627.864807] .cpu_load[3] : 0 [Tue Mar 17 18:02:23 2020][8592627.869615] .cpu_load[4] : 0 [Tue Mar 17 18:02:23 2020][8592627.874409] .avg_idle : 8752144 [Tue Mar 17 18:02:23 2020][8592627.879721] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:23 2020][8592627.885048] [Tue Mar 17 18:02:23 2020][8592627.886723] cfs_rq[12]:/ [Tue Mar 17 18:02:23 2020][8592627.889435] .exec_clock : 0.000000 [Tue Mar 17 18:02:23 2020][8592627.894834] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:23 2020][8592627.900233] .min_vruntime : 328013290.920233 [Tue Mar 17 18:02:23 2020][8592627.906326] .max_vruntime : 0.000001 [Tue Mar 17 18:02:23 2020][8592627.911725] .spread : 0.000000 [Tue Mar 17 18:02:23 2020][8592627.917125] .spread0 : 10798149.201649 [Tue Mar 17 18:02:23 2020][8592627.923131] .nr_spread_over : 0 [Tue Mar 17 18:02:23 2020][8592627.927923] .nr_running : 0 [Tue Mar 17 18:02:23 2020][8592627.932715] .load : 0 [Tue Mar 17 18:02:23 2020][8592627.937509] .runnable_load_avg : 0 [Tue Mar 17 18:02:23 2020][8592627.942302] .blocked_load_avg : 0 [Tue Mar 17 18:02:23 2020][8592627.947093] .tg_load_avg : 0 [Tue Mar 17 18:02:23 2020][8592627.951888] .tg_load_contrib : 0 [Tue Mar 17 18:02:23 2020][8592627.956680] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:23 2020][8592627.961473] .tg->runnable_avg : 0 [Tue Mar 17 18:02:23 2020][8592627.966265] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:23 2020][8592627.971056] .throttled : 0 [Tue Mar 17 18:02:23 2020][8592627.975849] .throttle_count : 0 [Tue Mar 17 18:02:23 2020][8592627.980644] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:23 2020][8592627.985435] .avg->runnable_avg_period : 46824 [Tue Mar 17 18:02:23 2020][8592627.990577] [Tue Mar 17 18:02:23 2020][8592627.992247] rt_rq[12]:/ [Tue Mar 17 18:02:23 2020][8592627.994873] .rt_nr_running : 0 [Tue Mar 17 18:02:23 2020][8592627.999667] .rt_throttled : 0 [Tue Mar 17 18:02:23 2020][8592628.004458] .rt_time : 0.000000 [Tue Mar 17 18:02:23 2020][8592628.009858] .rt_runtime : 950.000000 [Tue Mar 17 18:02:23 2020][8592628.015432] [Tue Mar 17 18:02:23 2020][8592628.017103] runnable tasks: [Tue Mar 17 18:02:23 2020][8592628.020075] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:23 2020][8592628.031022] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:23 2020][8592628.041896] watchdog/12 71 -11.982087 2148155 0 0.000000 20155.493704 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.053226] migration/12 72 0.000000 117309 0 0.000000 82896.789342 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.064562] ksoftirqd/12 73 328013272.301792 564542650 120 0.000000 10732603.550808 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.075899] kworker/12:0H 75 3705.472309 10 100 0.000000 0.041348 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.087305] kdmwork-253:11 12602 328013154.202493 822321399 120 0.000000 30102652.930589 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.098644] kdmwork-253:16 12665 772452.521860 95961 120 0.000000 1042.069995 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.109979] kdmwork-253:19 12676 772418.975585 68226 120 0.000000 764.941434 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.121336] kworker/12:1H 18080 328013144.961920 1104191917 100 0.000000 7543847.030374 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.132775] kdmwork-253:84 18555 328013156.961883 809693959 120 0.000000 30446534.946349 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.144106] kdmwork-253:88 18596 328013144.965189 826836706 120 0.000000 29682209.207302 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.155448] kdmwork-253:101 19614 772418.791089 84660 120 0.000000 1039.654309 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.166814] ptlrpcd_00_06 64811 328012732.460276 4136824 120 0.000000 143438.444184 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.178184] ll_ost_io00_007 68373 328013144.992481 73745989 120 0.000000 5101463.279462 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.189530] ll_ost00_012 68484 320111148.061779 57161515 120 0.000000 1260602.862307 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.200867] ll_ost_io00_016 68493 328013145.037987 62848663 120 0.000000 4022504.587760 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.212219] ll_ost00_035 68633 328013153.387091 73104200 120 0.000000 1532682.505106 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.223565] ll_ost_io00_025 68730 328012813.333048 128066461 120 0.000000 8350571.273524 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.234903] ll_ost00_061 68761 312844737.984058 46018323 120 0.000000 974161.084331 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.246235] ll_ost_io00_028 68764 328013009.200281 76930353 120 0.000000 5280120.865028 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.257576] ll_ost00_066 68795 312844738.120094 32252989 120 0.000000 719830.569600 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.268907] ll_ost_io00_031 68801 328012722.432877 107559728 120 0.000000 7415748.501212 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.280244] ll_ost00_071 68812 312844737.863521 43200243 120 0.000000 909486.190605 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.291591] ll_ost_create00 69325 325479967.046107 411128 120 0.000000 4953.666680 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.302956] ldlm_bl_83 100164 328013278.928739 6112289 120 0.000000 83910.151073 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.314375] ldlm_cn00_011 111307 328011863.181128 19354505 120 0.000000 432802.852542 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.325802] ldlm_cn00_017 111458 328013196.243195 5892205 120 0.000000 139158.896191 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.337228] ldlm_cn00_019 30583 328011916.520598 23870651 120 0.000000 558582.920558 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.348586] ll_ost_io00_068 90865 328013151.873301 74629641 120 0.000000 5780622.500362 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.359925] ll_ost_io00_070 90871 328013033.357912 136443978 120 0.000000 8817517.641067 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.371266] ll_ost_io00_085 40240 328012859.763008 58592694 120 0.000000 3505556.404119 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.382599] ll_ost_io00_090 40246 328013144.975789 90345247 120 0.000000 5786918.141248 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.393932] ll_ost_io00_091 40247 328013144.992081 131298588 120 0.000000 9114236.220191 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.405274] ll_ost_io00_092 40792 328012850.902229 123903617 120 0.000000 8426761.181276 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.416637] ll_ost00_098 85267 312844737.855486 25506117 120 0.000000 589555.134475 0.000000 0 / [Tue Mar 17 18:02:23 2020][8592628.427978] ll_ost00_106 85275 312844737.850787 18294116 120 0.000000 399539.286057 0.000000 0 / [Tue Mar 17 18:02:24 2020][8592628.439325] kworker/12:1 124266 327644540.832654 707 120 0.000000 2.579027 0.000000 0 / [Tue Mar 17 18:02:24 2020][8592628.450750] kworker/12:2 4680 328013272.583045 41822 120 0.000000 616.071481 0.000000 0 / [Tue Mar 17 18:02:24 2020][8592628.462108] [Tue Mar 17 18:02:24 2020][8592628.463780] cpu#13, 1996.233 MHz [Tue Mar 17 18:02:24 2020][8592628.467190] .nr_running : 0 [Tue Mar 17 18:02:24 2020][8592628.471990] .load : 0 [Tue Mar 17 18:02:24 2020][8592628.476782] .nr_switches : 11756957202 [Tue Mar 17 18:02:24 2020][8592628.482455] .nr_load_updates : 2908773638 [Tue Mar 17 18:02:24 2020][8592628.488033] .nr_uninterruptible : -182551 [Tue Mar 17 18:02:24 2020][8592628.493355] .next_balance : 12887.272995 [Tue Mar 17 18:02:24 2020][8592628.499121] .curr->pid : 0 [Tue Mar 17 18:02:24 2020][8592628.503919] .clock : 8592628503.369556 [Tue Mar 17 18:02:24 2020][8592628.510097] .cpu_load[0] : 0 [Tue Mar 17 18:02:24 2020][8592628.514905] .cpu_load[1] : 0 [Tue Mar 17 18:02:24 2020][8592628.519715] .cpu_load[2] : 0 [Tue Mar 17 18:02:24 2020][8592628.524509] .cpu_load[3] : 0 [Tue Mar 17 18:02:24 2020][8592628.529303] .cpu_load[4] : 0 [Tue Mar 17 18:02:24 2020][8592628.534111] .avg_idle : 8776635 [Tue Mar 17 18:02:24 2020][8592628.539426] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:24 2020][8592628.544738] [Tue Mar 17 18:02:24 2020][8592628.546408] cfs_rq[13]:/ [Tue Mar 17 18:02:24 2020][8592628.549121] .exec_clock : 0.000000 [Tue Mar 17 18:02:24 2020][8592628.554519] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:24 2020][8592628.559918] .min_vruntime : 173496784.993029 [Tue Mar 17 18:02:24 2020][8592628.566010] .max_vruntime : 0.000001 [Tue Mar 17 18:02:24 2020][8592628.571410] .spread : 0.000000 [Tue Mar 17 18:02:24 2020][8592628.576810] .spread0 : -143718356.950271 [Tue Mar 17 18:02:24 2020][8592628.582990] .nr_spread_over : 0 [Tue Mar 17 18:02:24 2020][8592628.587780] .nr_running : 0 [Tue Mar 17 18:02:24 2020][8592628.592575] .load : 0 [Tue Mar 17 18:02:24 2020][8592628.597368] .runnable_load_avg : 0 [Tue Mar 17 18:02:24 2020][8592628.602161] .blocked_load_avg : 0 [Tue Mar 17 18:02:24 2020][8592628.606952] .tg_load_avg : 0 [Tue Mar 17 18:02:24 2020][8592628.611746] .tg_load_contrib : 0 [Tue Mar 17 18:02:24 2020][8592628.616537] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:24 2020][8592628.621331] .tg->runnable_avg : 0 [Tue Mar 17 18:02:24 2020][8592628.626124] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:24 2020][8592628.630915] .throttled : 0 [Tue Mar 17 18:02:24 2020][8592628.635710] .throttle_count : 0 [Tue Mar 17 18:02:24 2020][8592628.640500] .avg->runnable_avg_sum : 17 [Tue Mar 17 18:02:24 2020][8592628.645382] .avg->runnable_avg_period : 47392 [Tue Mar 17 18:02:24 2020][8592628.650523] [Tue Mar 17 18:02:24 2020][8592628.652192] rt_rq[13]:/ [Tue Mar 17 18:02:24 2020][8592628.654818] .rt_nr_running : 0 [Tue Mar 17 18:02:24 2020][8592628.659611] .rt_throttled : 0 [Tue Mar 17 18:02:24 2020][8592628.664405] .rt_time : 0.000000 [Tue Mar 17 18:02:24 2020][8592628.669804] .rt_runtime : 950.000000 [Tue Mar 17 18:02:24 2020][8592628.675376] [Tue Mar 17 18:02:24 2020][8592628.677050] runnable tasks: [Tue Mar 17 18:02:24 2020][8592628.680021] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:24 2020][8592628.690967] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:24 2020][8592628.701842] watchdog/13 76 -11.979170 2148155 0 0.000000 20283.835004 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.713181] migration/13 77 0.000000 71255 0 0.000000 72575.148140 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.724517] ksoftirqd/13 78 173496606.400085 465933011 120 0.000000 8986703.630028 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.735853] kworker/13:0H 80 4976.282821 10 100 0.000000 0.068491 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.747292] kdmwork-253:71 18390 173496625.880299 814155271 120 0.000000 27273508.395963 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.758642] kdmwork-253:100 19392 173496768.069590 806434623 120 0.000000 29425752.571613 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.769980] kdmwork-253:107 21029 173496619.489852 829975087 120 0.000000 29476975.334361 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.781319] kworker/13:1H 25210 173496621.964227 939091355 100 0.000000 6234425.426891 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.792665] sssd_nss 50266 173496770.918793 3407552 120 0.000000 111195.771322 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.804000] sssd_pam 50267 173496772.379965 1718159 120 0.000000 43184.125703 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.815340] qmgr 51242 173496759.984224 408540 120 0.000000 30273.444112 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.826678] kiblnd_sd_01_00 64786 173496785.181913 6693084698 120 0.000000 50992517.657527 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.838110] ptlrpcd_01_11 64828 173496759.801289 4554243 120 0.000000 135287.099176 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.849463] ll_ost_create01 66926 173403670.019105 3341967 120 0.000000 55552.833534 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.860799] ll_ost_create01 66930 173259574.812320 2366045 120 0.000000 39740.095046 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.872146] lquota_wb_fir-O 68198 173496737.101058 177413 120 0.000000 3578.120124 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.883507] ll_ost01_018 68540 173496226.043130 26421387 120 0.000000 653790.244239 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.894863] ll_ost01_057 68690 167752853.843616 19744351 120 0.000000 556162.259587 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.906198] ll_ost01_058 68693 167752853.864805 18749685 120 0.000000 454932.862820 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.917534] ll_ost_io01_018 68701 173496425.700103 117856246 120 0.000000 7306617.451543 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.928873] ll_ost01_065 68734 167752853.840219 24813192 120 0.000000 645953.792935 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.940209] ll_ost_io01_026 68768 173496253.384473 27470648 120 0.000000 2049106.284906 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.951546] ll_ost01_072 68791 167752853.838506 20991281 120 0.000000 538941.179284 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.962880] ll_ost01_077 68815 144166494.818944 9724435 120 0.000000 275647.863155 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.974214] ll_ost_io01_030 68822 173496242.542589 44199392 120 0.000000 3787205.997925 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.985549] ll_ost01_079 68826 172717242.307026 18611471 120 0.000000 484827.691747 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592628.996890] ll_ost_io01_039 68857 173496291.385149 74289548 120 0.000000 5054530.598725 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.008228] ll_ost_io01_054 68896 173496383.802685 38399931 120 0.000000 2648976.408773 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.019565] ll_ost_io01_059 71825 173496383.758261 67726621 120 0.000000 4574959.821417 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.030905] ll_ost_io01_062 73693 173496248.381206 24947175 120 0.000000 1635795.868463 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.042241] ll_ost_io01_070 73701 173496383.943079 30057408 120 0.000000 2153404.957957 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.053576] ll_ost_io01_074 73705 173496615.229494 84704656 120 0.000000 5958563.901177 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.064912] ll_ost_io01_077 73708 173496248.355076 34213875 120 0.000000 2288205.754795 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.076248] ll_ost_io01_078 73709 173496544.069342 60233691 120 0.000000 4862304.359299 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.087583] ll_ost_io01_082 73713 173496281.337895 71013617 120 0.000000 4751981.295196 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.098921] ll_ost_io01_085 73716 173496227.368063 26336849 120 0.000000 2058637.108776 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.110256] ll_ost_io01_088 73719 173496383.849201 24009012 120 0.000000 1799692.563482 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.121610] ldlm_bl_55 100136 173496772.139591 6112862 120 0.000000 83820.428188 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.133032] ldlm_bl_60 100141 173496771.771033 6115537 120 0.000000 83836.698055 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.144459] ldlm_bl_78 100159 173496772.989953 6110308 120 0.000000 83602.051514 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.155888] ldlm_cn01_027 111455 173495627.878540 10422629 120 0.000000 224331.930813 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.167340] ll_ost_io01_096 40785 173496076.327116 92097872 120 0.000000 5940987.326551 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.178671] ll_ost_io01_099 40790 173496250.001198 59467569 120 0.000000 4451736.841228 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.190009] ll_ost_io01_101 40804 173496067.592336 53885706 120 0.000000 3678922.886326 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.201343] ll_ost_io01_102 40808 173496037.418927 61549185 120 0.000000 3812408.880117 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.212680] ll_ost_io01_106 40817 173496109.607143 14507986 120 0.000000 895053.350682 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.224028] ll_ost01_103 101313 173496536.299335 24217723 120 0.000000 658438.935724 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.235450] ll_ost01_108 101320 144063324.284214 13814261 120 0.000000 364302.579959 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.246896] kworker/13:0 110312 173496757.286353 568981 120 0.000000 10743.197400 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.258317] crond 122971 169172316.956035 6 120 0.000000 3.550638 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.269754] kworker/13:1 12164 173399167.905104 2 120 0.000000 0.009898 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.281085] kworker/u258:0 12753 173496719.768994 274 120 0.000000 96.315252 0.000000 1 / [Tue Mar 17 18:02:24 2020][8592629.292480] [Tue Mar 17 18:02:24 2020][8592629.294157] cpu#14, 1996.233 MHz [Tue Mar 17 18:02:24 2020][8592629.297564] .nr_running : 0 [Tue Mar 17 18:02:24 2020][8592629.302356] .load : 0 [Tue Mar 17 18:02:24 2020][8592629.307151] .nr_switches : 17660017533 [Tue Mar 17 18:02:24 2020][8592629.312823] .nr_load_updates : 3745419567 [Tue Mar 17 18:02:24 2020][8592629.318401] .nr_uninterruptible : 14933762 [Tue Mar 17 18:02:24 2020][8592629.323799] .next_balance : 12887.273827 [Tue Mar 17 18:02:24 2020][8592629.329560] .curr->pid : 0 [Tue Mar 17 18:02:24 2020][8592629.334356] .clock : 8592629333.376273 [Tue Mar 17 18:02:24 2020][8592629.340537] .cpu_load[0] : 0 [Tue Mar 17 18:02:24 2020][8592629.345350] .cpu_load[1] : 0 [Tue Mar 17 18:02:24 2020][8592629.350146] .cpu_load[2] : 0 [Tue Mar 17 18:02:24 2020][8592629.354954] .cpu_load[3] : 0 [Tue Mar 17 18:02:24 2020][8592629.359762] .cpu_load[4] : 0 [Tue Mar 17 18:02:24 2020][8592629.364558] .avg_idle : 3653062 [Tue Mar 17 18:02:24 2020][8592629.369885] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:24 2020][8592629.375199] [Tue Mar 17 18:02:24 2020][8592629.376871] cfs_rq[14]:/ [Tue Mar 17 18:02:24 2020][8592629.379582] .exec_clock : 0.000000 [Tue Mar 17 18:02:24 2020][8592629.384983] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:24 2020][8592629.390381] .min_vruntime : 203425971.580549 [Tue Mar 17 18:02:24 2020][8592629.396473] .max_vruntime : 0.000001 [Tue Mar 17 18:02:24 2020][8592629.401872] .spread : 0.000000 [Tue Mar 17 18:02:24 2020][8592629.407272] .spread0 : -113789170.580661 [Tue Mar 17 18:02:24 2020][8592629.413452] .nr_spread_over : 0 [Tue Mar 17 18:02:24 2020][8592629.418245] .nr_running : 0 [Tue Mar 17 18:02:25 2020][8592629.423038] .load : 0 [Tue Mar 17 18:02:25 2020][8592629.427830] .runnable_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592629.432624] .blocked_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592629.437415] .tg_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592629.442209] .tg_load_contrib : 0 [Tue Mar 17 18:02:25 2020][8592629.447001] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:25 2020][8592629.451794] .tg->runnable_avg : 0 [Tue Mar 17 18:02:25 2020][8592629.456587] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:25 2020][8592629.461380] .throttled : 0 [Tue Mar 17 18:02:25 2020][8592629.466171] .throttle_count : 0 [Tue Mar 17 18:02:25 2020][8592629.470965] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:25 2020][8592629.475758] .avg->runnable_avg_period : 47791 [Tue Mar 17 18:02:25 2020][8592629.480898] [Tue Mar 17 18:02:25 2020][8592629.482569] rt_rq[14]:/ [Tue Mar 17 18:02:25 2020][8592629.485195] .rt_nr_running : 0 [Tue Mar 17 18:02:25 2020][8592629.489989] .rt_throttled : 0 [Tue Mar 17 18:02:25 2020][8592629.494780] .rt_time : 0.000000 [Tue Mar 17 18:02:25 2020][8592629.500180] .rt_runtime : 950.000000 [Tue Mar 17 18:02:25 2020][8592629.505752] [Tue Mar 17 18:02:25 2020][8592629.507426] runnable tasks: [Tue Mar 17 18:02:25 2020][8592629.510399] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:25 2020][8592629.521343] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:25 2020][8592629.532203] systemd 1 203425962.955247 3414546 120 0.000000 435181.299634 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.543555] watchdog/14 81 -11.978720 2148157 0 0.000000 22387.108978 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.554892] migration/14 82 0.000000 116473 0 0.000000 183459.282247 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.566228] ksoftirqd/14 83 203424483.076037 446915463 120 0.000000 6125719.689155 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.577564] kworker/14:0H 85 80801541.509381 12 100 0.000000 0.061255 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.588996] kdmwork-253:50 18067 203424520.646607 876400675 120 0.000000 27952496.253897 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.600346] kdmwork-253:80 18517 629252.881822 88165 120 0.000000 1062.832674 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.611689] kdmwork-253:103 19994 629252.812576 83138 120 0.000000 1020.840831 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.623031] kdmwork-253:115 27526 203424520.369224 804916297 120 0.000000 29297277.256202 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.634366] kdmwork-253:116 27774 203425959.608301 825459061 120 0.000000 29588126.139646 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.645705] auditd 50140 203425311.877974 98516 116 0.000000 2437.752579 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.657043] sssd_be 50264 203425959.668824 5193682 120 0.000000 6063152.932468 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.668377] rsyslogd 50882 203422594.741210 14400 120 0.000000 448.281655 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.679713] writer#2 50896 203425961.418085 49028534 120 0.000000 721802.568075 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.691050] cfs_rh_00 64739 203288817.861110 10484 120 0.000000 4448.569526 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.702400] ptlrpcd_02_11 64840 203424263.809179 2540307 120 0.000000 98713.693349 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.713748] ldlm_cb02_000 66879 202751815.486431 1811 120 0.000000 36.488094 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.725098] ll_ost02_007 68205 203424489.756294 42428664 120 0.000000 1231863.183291 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.736445] ldlm_cn02_003 68378 203423919.452580 32626091 120 0.000000 760518.570571 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.747788] ll_ost02_017 68480 203424345.164990 31586379 120 0.000000 898737.182218 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.759131] ll_ost_io02_019 68506 203424241.588704 113361942 120 0.000000 7288424.770313 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.770483] ll_ost02_032 68594 203424613.396178 41564920 120 0.000000 1169548.360318 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.781813] ll_ost02_033 68595 203424524.426079 48292265 120 0.000000 1396898.060407 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.793165] ll_ost02_064 68745 202074888.050378 46190284 120 0.000000 1311189.065700 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.804521] ldlm_bl_04 69224 203425959.622037 6338068 120 0.000000 87445.289790 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.815876] ll_ost02_081 98672 203424477.851314 49621650 120 0.000000 1423767.583001 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.827211] ll_ost02_083 98674 203425960.010560 46534841 120 0.000000 1353022.663656 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.838548] ll_ost02_087 98678 202074966.775992 50458516 120 0.000000 1426349.631365 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.849883] ll_ost02_090 98681 203424345.890065 46367497 120 0.000000 1309528.063335 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.861219] ll_ost02_096 98687 203424557.211263 52526764 120 0.000000 1463048.351164 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.872564] ldlm_bl_58 100139 203425959.617989 6117517 120 0.000000 84219.335060 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.883985] ldlm_bl_59 100140 203425959.631104 6117598 120 0.000000 84125.823044 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.895417] grep 102170 203425971.445922 83694 120 0.000000 826.572103 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.906844] ldlm_cn02_011 111310 203423915.541843 25912881 120 0.000000 627990.918609 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.918265] ldlm_cn02_014 111320 203423915.541663 13933858 120 0.000000 339968.430429 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.929695] ldlm_cn02_023 64116 203425962.071409 19345802 120 0.000000 459628.050949 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.941039] ll_ost_io02_062 87184 203424274.583603 130967876 120 0.000000 8357598.842766 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.952380] ll_ost_io02_070 90839 203424495.112426 150934223 120 0.000000 9997709.497205 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.963716] ll_ost_io02_080 90855 203424403.281445 108235840 120 0.000000 7073209.373326 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.975070] ll_ost_io02_110 40836 203424320.567047 70230019 120 0.000000 4447787.608813 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.986414] ll_ost02_105 101336 203424492.468843 32572808 120 0.000000 923765.004387 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592629.997833] ll_ost02_106 101338 203424397.806235 36435610 120 0.000000 1042876.969379 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.009254] ll_ost02_107 101340 199198857.558474 35475125 120 0.000000 1040479.143963 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.020686] ll_ost02_108 85261 202074888.051119 21618127 120 0.000000 620094.631277 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.032023] ll_ost02_109 85263 203424450.357590 24632162 120 0.000000 702382.410618 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.043359] ll_ost02_110 85265 202074888.050067 20198663 120 0.000000 583996.110996 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.054702] kworker/14:2H 40930 203425480.431295 762508375 100 0.000000 4531673.114497 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.066043] md0_resync 55899 203424236.000324 435105003 125 0.000000 19079943.153873 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.077377] md6_resync 24408 203424227.725424 271447532 125 0.000000 11371151.041776 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.088720] kworker/14:2 125105 203425959.606378 121963 120 0.000000 2429.122305 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.100144] kworker/14:0 2253 203196274.993990 14 120 0.000000 0.102493 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.111489] kworker/u259:2 13272 203425480.465797 860 120 0.000000 40.379023 0.000000 2 / [Tue Mar 17 18:02:25 2020][8592630.122868] [Tue Mar 17 18:02:25 2020][8592630.124542] cpu#15, 1996.233 MHz [Tue Mar 17 18:02:25 2020][8592630.127950] .nr_running : 0 [Tue Mar 17 18:02:25 2020][8592630.132743] .load : 0 [Tue Mar 17 18:02:25 2020][8592630.137535] .nr_switches : 13732049466 [Tue Mar 17 18:02:25 2020][8592630.143209] .nr_load_updates : 3290871590 [Tue Mar 17 18:02:25 2020][8592630.148786] .nr_uninterruptible : -175284 [Tue Mar 17 18:02:25 2020][8592630.154098] .next_balance : 12887.274655 [Tue Mar 17 18:02:25 2020][8592630.159858] .curr->pid : 0 [Tue Mar 17 18:02:25 2020][8592630.164654] .clock : 8592630164.374148 [Tue Mar 17 18:02:25 2020][8592630.170833] .cpu_load[0] : 0 [Tue Mar 17 18:02:25 2020][8592630.175653] .cpu_load[1] : 0 [Tue Mar 17 18:02:25 2020][8592630.180445] .cpu_load[2] : 0 [Tue Mar 17 18:02:25 2020][8592630.185237] .cpu_load[3] : 0 [Tue Mar 17 18:02:25 2020][8592630.190030] .cpu_load[4] : 0 [Tue Mar 17 18:02:25 2020][8592630.194837] .avg_idle : 10000000 [Tue Mar 17 18:02:25 2020][8592630.200237] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:25 2020][8592630.205551] [Tue Mar 17 18:02:25 2020][8592630.207221] cfs_rq[15]:/ [Tue Mar 17 18:02:25 2020][8592630.209934] .exec_clock : 0.000000 [Tue Mar 17 18:02:25 2020][8592630.215333] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:25 2020][8592630.220732] .min_vruntime : 178901630.863790 [Tue Mar 17 18:02:25 2020][8592630.226824] .max_vruntime : 0.000001 [Tue Mar 17 18:02:25 2020][8592630.232224] .spread : 0.000000 [Tue Mar 17 18:02:25 2020][8592630.237624] .spread0 : -138313511.841348 [Tue Mar 17 18:02:25 2020][8592630.243804] .nr_spread_over : 0 [Tue Mar 17 18:02:25 2020][8592630.248597] .nr_running : 0 [Tue Mar 17 18:02:25 2020][8592630.253388] .load : 0 [Tue Mar 17 18:02:25 2020][8592630.258182] .runnable_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592630.262973] .blocked_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592630.267766] .tg_load_avg : 0 [Tue Mar 17 18:02:25 2020][8592630.272560] .tg_load_contrib : 0 [Tue Mar 17 18:02:25 2020][8592630.277352] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:25 2020][8592630.282144] .tg->runnable_avg : 0 [Tue Mar 17 18:02:25 2020][8592630.286936] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:25 2020][8592630.291729] .throttled : 0 [Tue Mar 17 18:02:25 2020][8592630.296523] .throttle_count : 0 [Tue Mar 17 18:02:25 2020][8592630.301315] .avg->runnable_avg_sum : 56 [Tue Mar 17 18:02:25 2020][8592630.306194] .avg->runnable_avg_period : 46410 [Tue Mar 17 18:02:25 2020][8592630.311335] [Tue Mar 17 18:02:25 2020][8592630.313007] rt_rq[15]:/ [Tue Mar 17 18:02:25 2020][8592630.315633] .rt_nr_running : 0 [Tue Mar 17 18:02:25 2020][8592630.320426] .rt_throttled : 0 [Tue Mar 17 18:02:25 2020][8592630.325219] .rt_time : 0.000000 [Tue Mar 17 18:02:25 2020][8592630.330619] .rt_runtime : 1000.000000 [Tue Mar 17 18:02:25 2020][8592630.336277] [Tue Mar 17 18:02:25 2020][8592630.337949] runnable tasks: [Tue Mar 17 18:02:25 2020][8592630.340923] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:25 2020][8592630.351867] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:25 2020][8592630.362741] watchdog/15 86 -11.979412 2148155 0 0.000000 20378.438412 0.000000 3 / [Tue Mar 17 18:02:25 2020][8592630.374073] migration/15 87 0.000000 60768 0 0.000000 73266.911468 0.000000 3 / [Tue Mar 17 18:02:25 2020][8592630.385407] ksoftirqd/15 88 178901609.989342 548106803 120 0.000000 9368307.154725 0.000000 3 / [Tue Mar 17 18:02:25 2020][8592630.396745] kworker/15:0H 90 1229.885133 10 100 0.000000 0.034457 0.000000 3 / [Tue Mar 17 18:02:25 2020][8592630.408110] kswapd3 279 178900837.091921 2363138 120 0.000000 12711994.411461 0.000000 3 / [Tue Mar 17 18:02:25 2020][8592630.419447] mlx5_health0000 2111 375.303383 2 100 0.000000 0.020929 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.430788] kworker/15:1H 5600 178901610.023417 1030871623 100 0.000000 6399596.745838 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.442220] multipathd 17898 0.000000 8 0 0.000000 1.002448 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.453557] multipathd 17919 0.000000 4 0 0.000000 1.042333 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.464892] multipathd 17924 0.000000 5 0 0.000000 1.029038 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.476253] kdmwork-253:14 12656 178901610.035719 870765589 120 0.000000 31784809.766665 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.487596] kdmwork-253:26 12697 513633.131956 96172 120 0.000000 1034.333868 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.498939] kdmwork-253:33 12718 178901617.229629 871960283 120 0.000000 30811446.705899 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.510287] kdmwork-253:53 18170 513633.090487 95111 120 0.000000 998.704856 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.521629] kdmwork-253:62 18293 513633.015270 62265 120 0.000000 723.973236 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.532974] kdmwork-253:77 18489 178901622.023345 810507273 120 0.000000 30499601.638021 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.544307] kdmwork-253:82 18533 513632.803297 74610 120 0.000000 899.861001 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.555657] kdmwork-253:110 25234 513633.003322 63243 120 0.000000 748.231588 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.566999] kdmflush 27829 7072.722980 2 100 0.000000 0.012635 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.578353] ptlrpcd_03_07 64848 178900743.867848 3563374 120 0.000000 115270.466661 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.589705] ll_ost_create03 66940 178901618.976414 3799078 120 0.000000 61792.218179 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.601039] ll_ost_io03_001 66973 178901605.895342 99770387 120 0.000000 6142174.745635 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.612387] ll_ost_io03_004 68365 178900664.749944 93339523 120 0.000000 5319105.898420 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.623725] ldlm_cn03_003 68469 178901617.863817 11481029 120 0.000000 251033.176565 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.635072] ll_ost_create03 68536 178679330.848931 1974752 120 0.000000 32446.252418 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.646415] ll_ost03_025 68566 178901598.939921 21740338 120 0.000000 607994.802459 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.657755] ll_ost03_031 68589 178901610.170313 22447413 120 0.000000 668792.776941 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.669087] ll_ost03_033 68597 178312188.349879 15181453 120 0.000000 480914.889626 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.680431] ll_ost03_047 68643 178901611.502081 18218085 120 0.000000 537819.306496 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.691779] ll_ost_io03_024 68762 178900745.031918 182331448 120 0.000000 11098390.692520 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.703126] ll_ost_io03_034 68871 178901527.906681 72349660 120 0.000000 4387938.166165 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.714462] ll_ost_io03_044 68901 178901523.083072 115739502 120 0.000000 6948446.050975 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.725819] ldlm_bl_12 100093 178901618.874580 6116760 120 0.000000 84198.094867 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.737243] ldlm_bl_17 100098 178901618.872917 6107409 120 0.000000 83576.440829 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.748665] ldlm_bl_22 100103 178901618.872997 6107012 120 0.000000 83907.014602 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.760092] ldlm_bl_38 100119 178901618.870162 6107219 120 0.000000 83643.818751 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.771533] ldlm_cn03_019 64125 178900235.384504 12520524 120 0.000000 296180.042522 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.782867] ldlm_cn03_028 64134 178490071.290307 20904896 120 0.000000 472650.869750 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.794199] ldlm_cn03_029 64135 178899421.127603 30627506 120 0.000000 677419.410444 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.805540] ll_ost_io03_059 87197 178901610.102575 47058818 120 0.000000 2904683.609564 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.816872] ll_ost_io03_061 87199 178900704.269499 119728098 120 0.000000 6462557.547502 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.828214] ll_ost_io03_069 90860 178901613.144245 79821108 120 0.000000 4570690.504515 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.839561] ll_ost_io03_084 40797 178901577.103485 37881996 120 0.000000 2069643.757207 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.850907] ll_ost_io03_093 40832 178901434.110093 115069512 120 0.000000 6830655.947075 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.862261] ll_ost03_067 998 169766390.408865 19700715 120 0.000000 551504.932622 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.873599] ll_ost03_069 70315 178901575.064942 5026329 120 0.000000 154113.380144 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.884935] ll_ost03_079 70326 176179433.873288 6666416 120 0.000000 220056.852818 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.896268] ll_ost03_086 70525 178901624.089816 8494002 120 0.000000 259127.336983 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.907607] ll_ost03_094 74614 169766390.408715 11268391 120 0.000000 333495.232687 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.918951] ll_ost03_107 125740 178901629.943475 4040465 120 0.000000 123238.267204 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.930371] ll_ost03_110 125743 178901613.042375 14309045 120 0.000000 411823.306477 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.941806] kworker/15:2 77964 178851102.059050 48 120 0.000000 0.126135 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.953151] kworker/15:0 13144 178901618.870463 4144 120 0.000000 105.501795 0.000000 3 / [Tue Mar 17 18:02:26 2020][8592630.964529] [Tue Mar 17 18:02:26 2020][8592630.966201] cpu#16, 1996.233 MHz [Tue Mar 17 18:02:26 2020][8592630.969611] .nr_running : 0 [Tue Mar 17 18:02:26 2020][8592630.974404] .load : 0 [Tue Mar 17 18:02:26 2020][8592630.979195] .nr_switches : 22654291506 [Tue Mar 17 18:02:26 2020][8592630.984871] .nr_load_updates : 3919110130 [Tue Mar 17 18:02:26 2020][8592630.990446] .nr_uninterruptible : -742871 [Tue Mar 17 18:02:26 2020][8592630.995757] .next_balance : 12887.275499 [Tue Mar 17 18:02:26 2020][8592631.001518] .curr->pid : 0 [Tue Mar 17 18:02:26 2020][8592631.006329] .clock : 8592631005.377738 [Tue Mar 17 18:02:26 2020][8592631.012511] .cpu_load[0] : 0 [Tue Mar 17 18:02:26 2020][8592631.017318] .cpu_load[1] : 0 [Tue Mar 17 18:02:26 2020][8592631.022114] .cpu_load[2] : 0 [Tue Mar 17 18:02:26 2020][8592631.026906] .cpu_load[3] : 0 [Tue Mar 17 18:02:26 2020][8592631.031711] .cpu_load[4] : 0 [Tue Mar 17 18:02:26 2020][8592631.036509] .avg_idle : 10000000 [Tue Mar 17 18:02:26 2020][8592631.041908] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:26 2020][8592631.047237] [Tue Mar 17 18:02:26 2020][8592631.048907] cfs_rq[16]:/ [Tue Mar 17 18:02:26 2020][8592631.051621] .exec_clock : 0.000000 [Tue Mar 17 18:02:26 2020][8592631.057019] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:26 2020][8592631.062419] .min_vruntime : 303807706.198406 [Tue Mar 17 18:02:26 2020][8592631.068512] .max_vruntime : 0.000001 [Tue Mar 17 18:02:26 2020][8592631.073911] .spread : 0.000000 [Tue Mar 17 18:02:26 2020][8592631.079311] .spread0 : -13407436.828409 [Tue Mar 17 18:02:26 2020][8592631.085402] .nr_spread_over : 0 [Tue Mar 17 18:02:26 2020][8592631.090196] .nr_running : 0 [Tue Mar 17 18:02:26 2020][8592631.094987] .load : 0 [Tue Mar 17 18:02:26 2020][8592631.099782] .runnable_load_avg : 0 [Tue Mar 17 18:02:26 2020][8592631.104573] .blocked_load_avg : 0 [Tue Mar 17 18:02:26 2020][8592631.109367] .tg_load_avg : 0 [Tue Mar 17 18:02:26 2020][8592631.114159] .tg_load_contrib : 0 [Tue Mar 17 18:02:26 2020][8592631.118952] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:26 2020][8592631.123745] .tg->runnable_avg : 0 [Tue Mar 17 18:02:26 2020][8592631.128536] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:26 2020][8592631.133330] .throttled : 0 [Tue Mar 17 18:02:26 2020][8592631.138123] .throttle_count : 0 [Tue Mar 17 18:02:26 2020][8592631.142916] .avg->runnable_avg_sum : 86 [Tue Mar 17 18:02:26 2020][8592631.147795] .avg->runnable_avg_period : 47708 [Tue Mar 17 18:02:26 2020][8592631.152935] [Tue Mar 17 18:02:26 2020][8592631.154607] rt_rq[16]:/ [Tue Mar 17 18:02:26 2020][8592631.157234] .rt_nr_running : 0 [Tue Mar 17 18:02:26 2020][8592631.162025] .rt_throttled : 0 [Tue Mar 17 18:02:26 2020][8592631.166817] .rt_time : 0.006432 [Tue Mar 17 18:02:26 2020][8592631.172216] .rt_runtime : 1000.000000 [Tue Mar 17 18:02:26 2020][8592631.177878] [Tue Mar 17 18:02:26 2020][8592631.179549] runnable tasks: [Tue Mar 17 18:02:26 2020][8592631.182522] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:26 2020][8592631.193467] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:26 2020][8592631.204344] watchdog/16 91 -11.982387 2148156 0 0.000000 20792.028428 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.215682] migration/16 92 0.000000 109118 0 0.000000 103748.085854 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.227017] ksoftirqd/16 93 303807661.118958 606011592 120 0.000000 12433206.264807 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.238355] kworker/16:0H 95 6928.800691 10 100 0.000000 0.122920 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.249719] scsi_eh_2 2671 3843.494038 21 120 0.000000 0.125168 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.261084] kdmwork-253:4 12488 861044.370748 68537 120 0.000000 769.166770 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.272431] kdmwork-253:20 12679 861044.354460 68264 120 0.000000 787.734619 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.283789] kdmwork-253:49 18062 861085.199292 94154 120 0.000000 995.898642 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.295122] kworker/16:1H 18071 303807634.663458 1204786035 100 0.000000 8527489.036929 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.306580] reader#1 50900 303807692.907651 1063204 120 0.000000 181382.709782 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.317929] ptlrpcd_00_09 64814 303807079.617921 4206617 120 0.000000 137647.280044 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.329278] kmmpd-md6 66363 303807030.747344 19048284 120 0.000000 153877.655377 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.340614] ldlm_cb00_000 66873 268960117.702688 12 120 0.000000 0.162564 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.351949] ldlm_cn00_000 66883 303807668.276105 29853753 120 0.000000 665610.316840 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.363295] ll_ost00_004 67819 295303356.471561 65595029 120 0.000000 1372329.829063 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.374634] lquota_wb_fir-O 68280 303807692.532185 174142 120 0.000000 3716.609254 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.385974] ll_ost_io00_009 68382 303807569.883692 89844945 120 0.000000 6155892.578398 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.397331] ll_ost00_027 68596 295303356.474686 47851384 120 0.000000 994423.641954 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.408665] ll_ost00_029 68603 295303356.496297 48166346 120 0.000000 1073137.605560 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.420004] ll_ost00_034 68629 303807700.706993 48220514 120 0.000000 1072823.503306 0.000000 0 / [Tue Mar 17 18:02:26 2020][8592631.431343] ll_ost_io00_021 68712 303807059.858234 123935253 120 0.000000 8256968.541379 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.442681] ll_ost_io00_024 68723 303807289.679964 78294974 120 0.000000 5363948.233143 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.454017] ll_ost00_057 68729 295303356.638635 57427758 120 0.000000 1230255.162674 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.465360] ll_ost00_065 68794 295303356.525612 43113241 120 0.000000 943839.526634 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.476699] ll_ost00_070 68808 303807066.149520 57613985 120 0.000000 1237101.158266 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.488040] ll_ost_io00_040 68870 303807557.993461 91930208 120 0.000000 6703770.441668 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.499380] ll_ost_io00_041 68874 303807694.389184 131078412 120 0.000000 8039931.133485 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.510716] ll_ost_io00_048 68895 303807218.807433 39914067 120 0.000000 2692456.177113 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.522061] ldlm_cn00_007 72312 303806723.720037 7489460 120 0.000000 174293.097359 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.533403] ldlm_cn00_008 88696 303806725.253575 20203942 120 0.000000 446326.568179 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.544750] ldlm_bl_11 100092 303807694.212553 6115248 120 0.000000 84315.187248 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.556180] ldlm_bl_76 100157 303807694.206672 6111679 120 0.000000 83949.788152 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.567605] ldlm_bl_90 100171 303807694.206411 6115996 120 0.000000 83897.568859 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.579025] ldlm_bl_98 100179 303807694.210528 6108157 120 0.000000 83583.400683 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.590455] ldlm_cn00_014 111321 303806723.719535 14204940 120 0.000000 306594.482727 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.601884] ldlm_cn00_026 30590 303807694.612245 9120845 120 0.000000 207460.041390 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.613217] ldlm_cn00_029 30593 303807627.843464 28359156 120 0.000000 665721.154428 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.624566] ll_ost_io00_055 87200 303807296.485712 65944259 120 0.000000 4377645.361408 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.635907] ll_ost_io00_077 40232 303807522.237228 88954318 120 0.000000 5573184.629135 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.647241] ll_ost_io00_079 40234 303807222.198840 40271383 120 0.000000 3084867.946524 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.658577] ll_ost_io00_082 40237 303807230.619590 96710475 120 0.000000 5991434.291853 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.669913] ll_ost_io00_087 40242 303807557.975717 140598419 120 0.000000 9371803.494308 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.681250] ll_ost_io00_088 40243 303807407.074060 145316504 120 0.000000 8995987.713782 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.692592] ll_ost_io00_094 40794 303807295.832364 111600279 120 0.000000 6958721.080655 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.703931] ll_ost_io00_095 40802 303807086.805253 93204154 120 0.000000 5790819.872195 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.715267] ll_ost_io00_096 40806 303807563.979626 45552174 120 0.000000 3528828.994449 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.726603] ll_ost_io00_097 40813 303807425.608988 121202864 120 0.000000 7438423.488181 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.737938] ll_ost_io00_098 40821 303807279.747800 105335866 120 0.000000 6672054.621447 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.749278] ll_ost_io00_102 40838 303807395.941606 80989117 120 0.000000 5335011.834658 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.760625] ll_ost00_084 101333 295303356.480487 32685540 120 0.000000 712825.402713 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.772050] ll_ost00_090 74374 295303356.690603 32102512 120 0.000000 712646.726274 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.783388] ll_ost00_092 85257 303807694.207503 46261497 120 0.000000 1050518.918333 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.794727] ll_ost00_110 125735 303807692.546753 10041879 120 0.000000 223947.122096 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.806168] kworker/16:2 13062 303785691.861021 9 120 0.000000 0.228599 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.817502] kworker/16:0 13721 303807694.520393 4067 120 0.000000 108.551117 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.828834] kworker/u262:0 13913 303807692.548536 478 100 0.000000 3.739760 0.000000 0 / [Tue Mar 17 18:02:27 2020][8592631.840216] [Tue Mar 17 18:02:27 2020][8592631.841888] cpu#17, 1996.233 MHz [Tue Mar 17 18:02:27 2020][8592631.845297] .nr_running : 0 [Tue Mar 17 18:02:27 2020][8592631.850088] .load : 0 [Tue Mar 17 18:02:27 2020][8592631.854883] .nr_switches : 12033684811 [Tue Mar 17 18:02:27 2020][8592631.860557] .nr_load_updates : 2954628797 [Tue Mar 17 18:02:27 2020][8592631.866131] .nr_uninterruptible : -179656 [Tue Mar 17 18:02:27 2020][8592631.871445] .next_balance : 12887.276375 [Tue Mar 17 18:02:27 2020][8592631.877204] .curr->pid : 0 [Tue Mar 17 18:02:27 2020][8592631.882001] .clock : 8592631881.384146 [Tue Mar 17 18:02:27 2020][8592631.888181] .cpu_load[0] : 0 [Tue Mar 17 18:02:27 2020][8592631.892987] .cpu_load[1] : 0 [Tue Mar 17 18:02:27 2020][8592631.897783] .cpu_load[2] : 0 [Tue Mar 17 18:02:27 2020][8592631.902574] .cpu_load[3] : 0 [Tue Mar 17 18:02:27 2020][8592631.907369] .cpu_load[4] : 0 [Tue Mar 17 18:02:27 2020][8592631.912191] .avg_idle : 10000000 [Tue Mar 17 18:02:27 2020][8592631.917596] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:27 2020][8592631.922907] [Tue Mar 17 18:02:27 2020][8592631.924577] cfs_rq[17]:/ [Tue Mar 17 18:02:27 2020][8592631.927289] .exec_clock : 0.000000 [Tue Mar 17 18:02:27 2020][8592631.932690] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:27 2020][8592631.938088] .min_vruntime : 175935107.118326 [Tue Mar 17 18:02:27 2020][8592631.944181] .max_vruntime : 0.000001 [Tue Mar 17 18:02:27 2020][8592631.949581] .spread : 0.000000 [Tue Mar 17 18:02:27 2020][8592631.954979] .spread0 : -141280036.152988 [Tue Mar 17 18:02:27 2020][8592631.961159] .nr_spread_over : 0 [Tue Mar 17 18:02:27 2020][8592631.965952] .nr_running : 0 [Tue Mar 17 18:02:27 2020][8592631.970744] .load : 0 [Tue Mar 17 18:02:27 2020][8592631.975537] .runnable_load_avg : 0 [Tue Mar 17 18:02:27 2020][8592631.980330] .blocked_load_avg : 0 [Tue Mar 17 18:02:27 2020][8592631.985123] .tg_load_avg : 0 [Tue Mar 17 18:02:27 2020][8592631.989916] .tg_load_contrib : 0 [Tue Mar 17 18:02:27 2020][8592631.994706] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:27 2020][8592631.999501] .tg->runnable_avg : 0 [Tue Mar 17 18:02:27 2020][8592632.004292] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:27 2020][8592632.009085] .throttled : 0 [Tue Mar 17 18:02:27 2020][8592632.013879] .throttle_count : 0 [Tue Mar 17 18:02:27 2020][8592632.018671] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:27 2020][8592632.023462] .avg->runnable_avg_period : 47879 [Tue Mar 17 18:02:27 2020][8592632.028605] [Tue Mar 17 18:02:27 2020][8592632.030276] rt_rq[17]:/ [Tue Mar 17 18:02:27 2020][8592632.032903] .rt_nr_running : 0 [Tue Mar 17 18:02:27 2020][8592632.037695] .rt_throttled : 0 [Tue Mar 17 18:02:27 2020][8592632.042488] .rt_time : 0.000000 [Tue Mar 17 18:02:27 2020][8592632.047887] .rt_runtime : 950.000000 [Tue Mar 17 18:02:27 2020][8592632.053459] [Tue Mar 17 18:02:27 2020][8592632.055130] runnable tasks: [Tue Mar 17 18:02:27 2020][8592632.058104] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:27 2020][8592632.069051] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:27 2020][8592632.079928] watchdog/17 96 -11.980043 2148155 0 0.000000 20243.811172 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.091265] migration/17 97 0.000000 75946 0 0.000000 73132.437672 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.102600] ksoftirqd/17 98 175934966.342229 469275215 120 0.000000 9045070.879803 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.113935] kworker/17:0H 100 3226.377920 10 100 0.000000 0.050266 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.125295] kauditd 442 175935087.030503 1590326 120 0.000000 15184.633327 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.136671] kdmwork-253:9 12591 175935038.027311 872115585 120 0.000000 30451123.507787 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.148016] kdmwork-253:23 12688 1022482.847650 227511 120 0.000000 3211.213744 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.159354] kdmwork-253:29 12706 1022389.086453 70321 120 0.000000 801.679142 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.170688] kdmwork-253:34 12721 1022389.073580 68104 120 0.000000 783.015527 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.182025] kdmwork-253:39 12756 175935038.027771 870326533 120 0.000000 28184093.715229 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.193361] kworker/17:1H 14294 175935038.011664 941595141 100 0.000000 6314963.809405 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.204707] kdmwork-253:54 18181 1022482.826627 226283 120 0.000000 3070.988027 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.216059] kdmwork-253:96 19146 175935050.011600 804980644 120 0.000000 29125360.279110 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.227409] crond 50991 175935087.074897 377921 120 0.000000 58392.906516 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.238759] ptlrpcd_01_02 64819 175935087.076500 4417738 120 0.000000 140523.219673 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.250107] ll_ost01_001 66909 170224802.005818 17320614 120 0.000000 438731.478400 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.261449] kmmpd-md2 67554 175934408.065178 13563408 120 0.000000 112963.917741 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.272793] lquota_wb_fir-O 68226 175935095.084315 177583 120 0.000000 3656.090275 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.284147] ll_ost_io01_016 68508 175934643.204226 82857609 120 0.000000 6122452.682049 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.295484] ll_ost01_019 68550 175934925.480185 22801935 120 0.000000 633615.682098 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.306826] ll_ost01_021 68564 175935084.957640 18956273 120 0.000000 531932.806974 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.318166] ll_ost01_034 68608 175934528.837179 20948677 120 0.000000 608125.231634 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.329504] ll_ost01_039 68619 175935087.561071 22850881 120 0.000000 644443.263364 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.340846] ll_ost01_048 68665 175934926.161968 17707134 120 0.000000 486285.854610 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.352186] ll_ost01_055 68686 170224801.982954 12984677 120 0.000000 350017.711962 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.363520] ll_ost01_056 68687 175934528.826641 24161704 120 0.000000 663690.830701 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.374858] ll_ost_io01_020 68709 175934904.709600 58637428 120 0.000000 4228465.374004 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.386195] ll_ost01_062 68720 175934528.956625 19198166 120 0.000000 539464.910520 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.397528] ll_ost_io01_022 68722 175934420.051452 99181864 120 0.000000 7251939.460111 0.000000 1 / [Tue Mar 17 18:02:27 2020][8592632.408872] ll_ost_io01_027 68782 175934429.082993 106582949 120 0.000000 6964815.484416 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.420215] ll_ost01_080 68827 147536945.329475 16113258 120 0.000000 413192.170801 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.431554] ll_ost01_081 68831 170224802.007650 24303305 120 0.000000 628573.205650 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.442892] ll_ost_io01_035 68848 175934427.375856 75978796 120 0.000000 4834169.071100 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.454229] ll_ost_io01_050 68884 175934387.519147 72849219 120 0.000000 4869679.173165 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.465566] ll_ost_create01 68990 175935095.257811 4289386 120 0.000000 74609.804242 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.476905] ll_ost_io01_072 73703 175934977.775532 62523927 120 0.000000 4579264.698903 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.488248] ll_ost_io01_090 73721 175934762.576355 44832945 120 0.000000 3075397.693541 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.499583] ll_ost01_086 97761 147536945.329205 14401449 120 0.000000 349025.020184 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.510916] ll_ost01_089 97768 175934643.601966 22996255 120 0.000000 633212.698074 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.522253] ll_ost01_090 97770 153918872.741782 17207942 120 0.000000 433470.329958 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.533596] ldlm_bl_49 100130 175935095.090978 6118031 120 0.000000 83558.916248 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.545019] ldlm_bl_51 100132 175935095.089455 6112544 120 0.000000 83936.761767 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.556452] ldlm_cn01_011 111297 175935095.309697 10667589 120 0.000000 239902.957723 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.567915] ll_ost_io01_100 40799 175934428.775576 80177556 120 0.000000 4635400.480773 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.579266] ll_ost01_091 101301 175935107.125610 17498945 120 0.000000 513811.471039 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.590685] ll_ost01_095 101305 175935077.058667 22221416 120 0.000000 621986.505258 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.602112] ll_ost01_110 101325 175934532.207576 16943826 120 0.000000 487534.407786 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.613560] kworker/17:2 4523 175935095.130159 54470 120 0.000000 918.104653 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.624894] kworker/17:0 9305 175721576.821528 2 120 0.000000 0.011001 0.000000 1 / [Tue Mar 17 18:02:28 2020][8592632.636304] [Tue Mar 17 18:02:28 2020][8592632.637980] cpu#18, 1996.233 MHz [Tue Mar 17 18:02:28 2020][8592632.641390] .nr_running : 0 [Tue Mar 17 18:02:28 2020][8592632.646189] .load : 0 [Tue Mar 17 18:02:28 2020][8592632.650982] .nr_switches : 18592947519 [Tue Mar 17 18:02:28 2020][8592632.656643] .nr_load_updates : 3843372439 [Tue Mar 17 18:02:28 2020][8592632.662252] .nr_uninterruptible : 13728886 [Tue Mar 17 18:02:28 2020][8592632.667656] .next_balance : 12887.277171 [Tue Mar 17 18:02:28 2020][8592632.673402] .curr->pid : 0 [Tue Mar 17 18:02:28 2020][8592632.678208] .clock : 8592632677.381677 [Tue Mar 17 18:02:28 2020][8592632.684394] .cpu_load[0] : 0 [Tue Mar 17 18:02:28 2020][8592632.689193] .cpu_load[1] : 0 [Tue Mar 17 18:02:28 2020][8592632.693986] .cpu_load[2] : 0 [Tue Mar 17 18:02:28 2020][8592632.698794] .cpu_load[3] : 0 [Tue Mar 17 18:02:28 2020][8592632.703590] .cpu_load[4] : 0 [Tue Mar 17 18:02:28 2020][8592632.708388] .avg_idle : 4246751 [Tue Mar 17 18:02:28 2020][8592632.713731] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:28 2020][8592632.719050] [Tue Mar 17 18:02:28 2020][8592632.720721] cfs_rq[18]:/ [Tue Mar 17 18:02:28 2020][8592632.723433] .exec_clock : 0.000000 [Tue Mar 17 18:02:28 2020][8592632.728831] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:28 2020][8592632.734230] .min_vruntime : 211768928.010971 [Tue Mar 17 18:02:28 2020][8592632.740323] .max_vruntime : 0.000001 [Tue Mar 17 18:02:28 2020][8592632.745724] .spread : 0.000000 [Tue Mar 17 18:02:28 2020][8592632.751122] .spread0 : -105446215.277937 [Tue Mar 17 18:02:28 2020][8592632.757302] .nr_spread_over : 0 [Tue Mar 17 18:02:28 2020][8592632.762095] .nr_running : 0 [Tue Mar 17 18:02:28 2020][8592632.766888] .load : 0 [Tue Mar 17 18:02:28 2020][8592632.771679] .runnable_load_avg : 0 [Tue Mar 17 18:02:28 2020][8592632.776473] .blocked_load_avg : 0 [Tue Mar 17 18:02:28 2020][8592632.781266] .tg_load_avg : 0 [Tue Mar 17 18:02:28 2020][8592632.786057] .tg_load_contrib : 0 [Tue Mar 17 18:02:28 2020][8592632.790851] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:28 2020][8592632.795642] .tg->runnable_avg : 0 [Tue Mar 17 18:02:28 2020][8592632.800436] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:28 2020][8592632.805228] .throttled : 0 [Tue Mar 17 18:02:28 2020][8592632.810022] .throttle_count : 0 [Tue Mar 17 18:02:28 2020][8592632.814813] .avg->runnable_avg_sum : 137 [Tue Mar 17 18:02:28 2020][8592632.819780] .avg->runnable_avg_period : 47601 [Tue Mar 17 18:02:28 2020][8592632.824923] [Tue Mar 17 18:02:28 2020][8592632.826592] rt_rq[18]:/ [Tue Mar 17 18:02:28 2020][8592632.829217] .rt_nr_running : 0 [Tue Mar 17 18:02:28 2020][8592632.834009] .rt_throttled : 0 [Tue Mar 17 18:02:28 2020][8592632.838804] .rt_time : 0.000000 [Tue Mar 17 18:02:28 2020][8592632.844203] .rt_runtime : 950.000000 [Tue Mar 17 18:02:28 2020][8592632.849774] [Tue Mar 17 18:02:28 2020][8592632.851448] runnable tasks: [Tue Mar 17 18:02:28 2020][8592632.854421] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:28 2020][8592632.865366] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:28 2020][8592632.876244] watchdog/18 101 -11.980163 2148157 0 0.000000 21923.539952 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.887579] migration/18 102 0.000000 123820 0 0.000000 203788.209939 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.898916] ksoftirqd/18 103 211765399.944165 494413749 120 0.000000 6878531.392809 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.910253] kworker/18:0H 105 153573741.627982 14 100 0.000000 0.102674 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.921649] kdmwork-253:3 12484 697856.451508 69244 120 0.000000 766.863815 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.933021] kdmwork-253:58 18243 697856.489560 96028 120 0.000000 1013.580868 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.944363] kdmwork-253:74 18418 211765422.256282 816236361 120 0.000000 29755766.303871 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.955716] kdmwork-253:106 20670 697856.740754 76163 120 0.000000 941.280206 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.967063] kdmflush 30889 7702.871239 2 100 0.000000 0.013215 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.978399] kdmwork-253:119 31154 697856.657672 62814 120 0.000000 702.771290 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592632.989739] in:imjournal 50889 211768928.010971 10335517 120 0.000000 358973.093290 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.001082] kiblnd_sd_02_00 64790 211768916.192172 11154281015 120 0.000000 88689189.184099 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.012602] jbd2/md10-8 66563 211765001.142781 14769708 120 0.000000 2950714.887329 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.023940] ll_ost02_000 66911 211765057.165065 42703383 120 0.000000 1211083.381638 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.035279] ll_ost_create02 66936 211768916.238809 3969313 120 0.000000 72454.829798 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.046619] ll_ost02_005 67744 211765458.975322 32763170 120 0.000000 919663.249202 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.057957] ll_ost02_009 68341 211765409.036173 39434753 120 0.000000 1119880.578353 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.069300] ll_ost02_012 68358 207513136.486849 44310790 120 0.000000 1213307.376334 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.080635] ll_ost_io02_010 68386 211765083.486887 147147684 120 0.000000 9477061.133743 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.091967] ll_ost_io02_011 68391 211765383.643879 71900556 120 0.000000 5133587.012557 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.103305] ll_ost02_013 68473 211765426.916351 39804483 120 0.000000 1085237.498684 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.114663] ll_ost02_047 68655 211765408.955141 33939257 120 0.000000 1008119.893615 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.126010] ll_ost02_061 68737 210411770.915956 43638975 120 0.000000 1241769.926017 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.137356] ll_ost02_078 68828 211765170.117377 35828286 120 0.000000 1005680.205266 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.148694] ll_ost_io02_037 68836 211765239.987881 121271074 120 0.000000 8313383.768006 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.160030] ll_ost_io02_047 68862 211765019.525036 122457575 120 0.000000 8049332.748478 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.171388] ll_ost_io02_055 97765 211765130.314835 96844389 120 0.000000 6084588.316993 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.182725] ll_ost_io02_056 97766 211765295.900966 117577314 120 0.000000 7516318.369518 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.194066] ll_ost_io02_061 98932 211765409.055923 72422502 120 0.000000 4718748.015554 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.205406] ldlm_bl_62 100143 211768913.411467 6117087 120 0.000000 83744.634655 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.216831] ldlm_bl_73 100154 211768913.413691 6116399 120 0.000000 84153.711905 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.228271] ldlm_cn02_028 64121 211768910.637612 21114534 120 0.000000 499246.070162 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.239613] ll_ost_io02_066 87193 211765186.260997 38724373 120 0.000000 2647823.963248 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.250988] ll_ost02_101 101329 211765365.178674 29975497 120 0.000000 860786.962832 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.262409] raid-check 57403 211766281.850202 1469847 120 0.000000 418891.324016 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.273759] kworker/18:1H 103791 211766266.046643 386217761 100 0.000000 2211215.622787 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.285198] kworker/18:2 4659 211768913.409994 33970 120 0.000000 556.868985 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.296531] kworker/18:1 11428 211644264.787506 2 120 0.000000 0.008776 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.307867] kworker/u259:1 12614 211765674.984597 5042 120 0.000000 488.657723 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.319204] kworker/u259:0 13851 211768899.726349 79 120 0.000000 2618.139501 0.000000 2 / [Tue Mar 17 18:02:28 2020][8592633.330564] [Tue Mar 17 18:02:28 2020][8592633.332237] cpu#19, 1996.233 MHz [Tue Mar 17 18:02:28 2020][8592633.335645] .nr_running : 0 [Tue Mar 17 18:02:28 2020][8592633.340437] .load : 0 [Tue Mar 17 18:02:28 2020][8592633.345232] .nr_switches : 13996935958 [Tue Mar 17 18:02:28 2020][8592633.350904] .nr_load_updates : 3302468304 [Tue Mar 17 18:02:28 2020][8592633.356498] .nr_uninterruptible : -184547 [Tue Mar 17 18:02:28 2020][8592633.361810] .next_balance : 12887.277863 [Tue Mar 17 18:02:28 2020][8592633.367569] .curr->pid : 0 [Tue Mar 17 18:02:28 2020][8592633.372367] .clock : 8592633371.387006 [Tue Mar 17 18:02:28 2020][8592633.378545] .cpu_load[0] : 0 [Tue Mar 17 18:02:28 2020][8592633.383352] .cpu_load[1] : 0 [Tue Mar 17 18:02:28 2020][8592633.388148] .cpu_load[2] : 0 [Tue Mar 17 18:02:28 2020][8592633.392942] .cpu_load[3] : 0 [Tue Mar 17 18:02:28 2020][8592633.397747] .cpu_load[4] : 0 [Tue Mar 17 18:02:28 2020][8592633.402545] .avg_idle : 9839095 [Tue Mar 17 18:02:28 2020][8592633.407865] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:28 2020][8592633.413206] [Tue Mar 17 18:02:28 2020][8592633.414882] cfs_rq[19]:/ [Tue Mar 17 18:02:28 2020][8592633.417594] .exec_clock : 0.000000 [Tue Mar 17 18:02:28 2020][8592633.422992] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:29 2020][8592633.428392] .min_vruntime : 187257163.221300 [Tue Mar 17 18:02:29 2020][8592633.434485] .max_vruntime : 0.000001 [Tue Mar 17 18:02:29 2020][8592633.439883] .spread : 0.000000 [Tue Mar 17 18:02:29 2020][8592633.445284] .spread0 : -129957980.071135 [Tue Mar 17 18:02:29 2020][8592633.451465] .nr_spread_over : 0 [Tue Mar 17 18:02:29 2020][8592633.456256] .nr_running : 0 [Tue Mar 17 18:02:29 2020][8592633.461049] .load : 0 [Tue Mar 17 18:02:29 2020][8592633.465841] .runnable_load_avg : 0 [Tue Mar 17 18:02:29 2020][8592633.470633] .blocked_load_avg : 0 [Tue Mar 17 18:02:29 2020][8592633.475427] .tg_load_avg : 0 [Tue Mar 17 18:02:29 2020][8592633.480220] .tg_load_contrib : 0 [Tue Mar 17 18:02:29 2020][8592633.485011] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:29 2020][8592633.489805] .tg->runnable_avg : 0 [Tue Mar 17 18:02:29 2020][8592633.494599] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:29 2020][8592633.499390] .throttled : 0 [Tue Mar 17 18:02:29 2020][8592633.504182] .throttle_count : 0 [Tue Mar 17 18:02:29 2020][8592633.508976] .avg->runnable_avg_sum : 8 [Tue Mar 17 18:02:29 2020][8592633.513768] .avg->runnable_avg_period : 46739 [Tue Mar 17 18:02:29 2020][8592633.518911] [Tue Mar 17 18:02:29 2020][8592633.520580] rt_rq[19]:/ [Tue Mar 17 18:02:29 2020][8592633.523206] .rt_nr_running : 0 [Tue Mar 17 18:02:29 2020][8592633.527999] .rt_throttled : 0 [Tue Mar 17 18:02:29 2020][8592633.532792] .rt_time : 0.000000 [Tue Mar 17 18:02:29 2020][8592633.538190] .rt_runtime : 950.000000 [Tue Mar 17 18:02:29 2020][8592633.543762] [Tue Mar 17 18:02:29 2020][8592633.545437] runnable tasks: [Tue Mar 17 18:02:29 2020][8592633.548408] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:29 2020][8592633.559354] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:29 2020][8592633.570234] watchdog/19 106 -11.980293 2148156 0 0.000000 18716.277806 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.581567] migration/19 107 0.000000 64934 0 0.000000 79896.401266 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.592904] ksoftirqd/19 108 187256953.509138 547909111 120 0.000000 9379246.312496 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.604239] kworker/19:0H 110 793.096968 10 100 0.000000 0.040544 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.615611] kworker/19:1H 5598 187256953.491130 1027242758 100 0.000000 6398024.741514 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.627083] kdmwork-253:35 12724 507079.762941 67994 120 0.000000 763.107091 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.638449] kdmwork-253:79 18507 507079.533952 86993 120 0.000000 1001.143928 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.649803] kdmwork-253:113 26036 507117.249339 74177 120 0.000000 919.071744 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.661158] kiblnd_sd_03_02 64796 187257163.320245 9108023275 120 0.000000 69247985.891010 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.672587] ptlrpcd_03_03 64844 187256221.985908 3582294 120 0.000000 116018.954958 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.683923] ptlrpcd_03_10 64851 187256221.982040 3441189 120 0.000000 119762.645958 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.695262] md0_raid6 66171 187256173.294302 1345668682 120 0.000000 460639620.462071 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.706688] ldlm_elt 66897 187257148.859569 268 120 0.000000 3973.391911 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.718034] jbd2/md2-8 67601 187256422.582588 14332562 120 0.000000 2872487.574036 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.729373] ll_ost03_005 68134 187256910.681912 17032327 120 0.000000 517061.161942 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.740718] ll_ost_io03_008 68390 187256555.012185 100774998 120 0.000000 6052693.605897 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.752064] ll_ost03_015 68535 187256958.820595 21566548 120 0.000000 649139.383902 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.763406] ll_ost03_039 68618 177679857.799171 21263356 120 0.000000 618343.529817 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.774744] ll_ost03_048 68646 177679857.798190 18235871 120 0.000000 528373.164242 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.786083] ll_ost03_058 68692 184345973.275482 15457289 120 0.000000 479544.488414 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.797422] ll_ost_io03_020 68708 187256678.508159 113715708 120 0.000000 6322894.059796 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.808763] ll_ost_io03_022 68748 187256732.728161 50123271 120 0.000000 3162687.718484 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.820094] ll_ost03_063 68749 184345973.273499 21818447 120 0.000000 620592.152272 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.831435] ll_ost_io03_027 68780 187256233.899495 127634489 120 0.000000 7413480.339380 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.842770] ll_ost_io03_029 68811 187256199.267805 107957397 120 0.000000 6155532.581976 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.854107] ll_ost_io03_032 68847 187256953.553943 105928262 120 0.000000 6179761.021721 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.865445] ll_ost_io03_045 68903 187256482.428147 64294246 120 0.000000 4056222.478261 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.876790] ldlm_bl_09 70868 187257151.038464 6327499 120 0.000000 87318.744619 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.888159] ldlm_bl_97 100178 187257150.493225 6106322 120 0.000000 83569.085599 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.899588] ldlm_cn03_013 31067 187256017.058150 21615824 120 0.000000 486211.220591 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.910931] ll_ost_io03_050 84743 187256447.212654 81323191 120 0.000000 5221874.228445 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.922268] ll_ost_io03_056 87191 187256220.078384 26564361 120 0.000000 1688883.945988 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.933603] ll_ost_io03_063 87203 187256954.057731 158907728 120 0.000000 9200394.132883 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.944942] ll_ost_io03_064 90844 187256927.471829 129853659 120 0.000000 7762813.581135 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.956278] ll_ost_io03_068 90858 187256961.661493 76518420 120 0.000000 4719003.988632 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.967613] ll_ost_io03_072 90869 187256953.538192 77837557 120 0.000000 4880058.255029 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.978967] ll_ost_io03_094 40834 187256952.154150 85197523 120 0.000000 4763932.163406 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592633.990317] ll_ost03_082 70329 187256967.911421 16153402 120 0.000000 453080.900968 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.001652] ll_ost03_085 70524 187256973.598461 17424795 120 0.000000 493476.973361 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.012989] ll_ost03_090 71243 187256954.523608 11415934 120 0.000000 335411.337168 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.024334] ll_ost03_108 125741 184345973.272387 13573943 120 0.000000 391792.894176 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.035758] ll_ost03_111 125744 187256954.614315 4376021 120 0.000000 146430.778483 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.047181] raid-check 55852 187257121.209638 164323 120 0.000000 22129.337545 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.058516] ll_ost_create03 10365 185191337.216504 371691 120 0.000000 6232.265159 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.069859] kworker/19:1 75494 187257140.602098 285516 120 0.000000 5919.721192 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.081196] kworker/19:0 124011 186510194.302604 2 120 0.000000 0.008987 0.000000 3 / [Tue Mar 17 18:02:29 2020][8592634.092659] [Tue Mar 17 18:02:29 2020][8592634.094328] cpu#20, 1996.233 MHz [Tue Mar 17 18:02:29 2020][8592634.097735] .nr_running : 0 [Tue Mar 17 18:02:29 2020][8592634.102528] .load : 0 [Tue Mar 17 18:02:29 2020][8592634.107321] .nr_switches : 17927208265 [Tue Mar 17 18:02:29 2020][8592634.112994] .nr_load_updates : 3381072119 [Tue Mar 17 18:02:29 2020][8592634.118571] .nr_uninterruptible : -912969 [Tue Mar 17 18:02:29 2020][8592634.123885] .next_balance : 12887.278626 [Tue Mar 17 18:02:29 2020][8592634.129653] .curr->pid : 0 [Tue Mar 17 18:02:29 2020][8592634.134449] .clock : 8592634134.389030 [Tue Mar 17 18:02:29 2020][8592634.140629] .cpu_load[0] : 0 [Tue Mar 17 18:02:29 2020][8592634.145436] .cpu_load[1] : 0 [Tue Mar 17 18:02:29 2020][8592634.150232] .cpu_load[2] : 0 [Tue Mar 17 18:02:29 2020][8592634.155039] .cpu_load[3] : 0 [Tue Mar 17 18:02:29 2020][8592634.159832] .cpu_load[4] : 0 [Tue Mar 17 18:02:29 2020][8592634.164640] .avg_idle : 7665929 [Tue Mar 17 18:02:29 2020][8592634.169956] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:29 2020][8592634.175270] [Tue Mar 17 18:02:29 2020][8592634.176937] cfs_rq[20]:/ [Tue Mar 17 18:02:29 2020][8592634.179651] .exec_clock : 0.000000 [Tue Mar 17 18:02:29 2020][8592634.185051] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:29 2020][8592634.190451] .min_vruntime : 324955517.291764 [Tue Mar 17 18:02:29 2020][8592634.196542] .max_vruntime : 0.000001 [Tue Mar 17 18:02:29 2020][8592634.201941] .spread : 0.000000 [Tue Mar 17 18:02:29 2020][8592634.207341] .spread0 : 7740373.999329 [Tue Mar 17 18:02:29 2020][8592634.213261] .nr_spread_over : 0 [Tue Mar 17 18:02:29 2020][8592634.218052] .nr_running : 0 [Tue Mar 17 18:02:29 2020][8592634.222844] .load : 0 [Tue Mar 17 18:02:29 2020][8592634.227638] .runnable_load_avg : 0 [Tue Mar 17 18:02:29 2020][8592634.232429] .blocked_load_avg : 1 [Tue Mar 17 18:02:29 2020][8592634.237224] .tg_load_avg : 0 [Tue Mar 17 18:02:29 2020][8592634.242016] .tg_load_contrib : 0 [Tue Mar 17 18:02:29 2020][8592634.246808] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:29 2020][8592634.251601] .tg->runnable_avg : 0 [Tue Mar 17 18:02:29 2020][8592634.256394] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:29 2020][8592634.261187] .throttled : 0 [Tue Mar 17 18:02:29 2020][8592634.265978] .throttle_count : 0 [Tue Mar 17 18:02:29 2020][8592634.270773] .avg->runnable_avg_sum : 98 [Tue Mar 17 18:02:29 2020][8592634.275651] .avg->runnable_avg_period : 46955 [Tue Mar 17 18:02:29 2020][8592634.280795] [Tue Mar 17 18:02:29 2020][8592634.282464] rt_rq[20]:/ [Tue Mar 17 18:02:29 2020][8592634.285090] .rt_nr_running : 0 [Tue Mar 17 18:02:29 2020][8592634.289881] .rt_throttled : 0 [Tue Mar 17 18:02:29 2020][8592634.294675] .rt_time : 0.000000 [Tue Mar 17 18:02:29 2020][8592634.300074] .rt_runtime : 950.000000 [Tue Mar 17 18:02:29 2020][8592634.305646] [Tue Mar 17 18:02:29 2020][8592634.307320] runnable tasks: [Tue Mar 17 18:02:29 2020][8592634.310292] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:29 2020][8592634.321238] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:29 2020][8592634.332117] watchdog/20 111 -11.982167 2148156 0 0.000000 19491.750597 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.343451] migration/20 112 0.000000 122643 0 0.000000 100315.840953 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.354786] ksoftirqd/20 113 324955502.791861 555458076 120 0.000000 10596403.855181 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.366123] kworker/20:0H 115 3036.702181 10 100 0.000000 0.058521 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.377507] multipathd 17948 0.000000 106557227 0 0.000000 13101672.169375 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.388852] kdmwork-253:1 12458 646514.566091 96221 120 0.000000 1015.210240 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.400195] kdmwork-253:17 12668 324955305.337302 876301293 120 0.000000 29196693.582892 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.411544] kworker/20:1H 14273 324955505.208517 1092310628 100 0.000000 7584468.057274 0.000000 0 / [Tue Mar 17 18:02:29 2020][8592634.422977] kdmwork-253:48 18035 324955306.749842 823706110 120 0.000000 28575387.155934 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.434326] kdmwork-253:67 18330 324955309.316801 813163119 120 0.000000 23441994.523640 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.445666] kdmwork-253:83 18543 646514.709648 62603 120 0.000000 698.444629 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.457025] sssd 50166 324955497.232273 286348 120 0.000000 8485.756055 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.468362] reader#0 50899 324955505.338645 11023023 120 0.000000 2083651.005108 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.479699] kiblnd_sd_00_03 64785 324955517.262306 11543884241 120 0.000000 74123514.649237 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.491214] ptlrpcd_00_00 64805 324954759.591938 4133971 120 0.000000 143472.076633 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.502560] kmmpd-md8 66745 324954734.141270 18536894 120 0.000000 146902.576415 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.513890] kmmpd-md0 66759 324954704.380524 17040490 120 0.000000 135760.999504 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.525268] ll_ost00_014 68546 315707379.092284 51189338 120 0.000000 1078529.992253 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.536608] ll_ost00_015 68547 319977720.411852 24165530 120 0.000000 508766.831252 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.547944] ll_ost00_016 68554 315707379.032832 47393474 120 0.000000 984144.502534 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.559279] ll_ost00_018 68560 315707379.134503 43941698 120 0.000000 946923.086178 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.570614] ll_ost00_019 68562 315707378.975313 52485984 120 0.000000 1151633.863132 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.581964] ll_ost00_038 68651 320270740.684067 19471262 120 0.000000 418538.817347 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.593300] ll_ost00_050 68689 324954721.021602 60636266 120 0.000000 1347427.558509 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.604643] ll_ost_io00_030 68783 324954749.890959 104871296 120 0.000000 7083672.473697 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.615981] ll_ost00_076 68821 315707379.041137 55197017 120 0.000000 1197489.723868 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.627325] ldlm_bl_08 70798 324955505.216380 6338542 120 0.000000 86921.076990 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.638670] ll_ost_create00 80079 323151569.390801 304480 120 0.000000 3775.827912 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.650016] ldlm_bl_46 100127 324955505.300350 6110972 120 0.000000 83844.719637 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.661434] ldlm_bl_52 100133 324955504.871836 6113502 120 0.000000 83934.735456 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.672860] ldlm_bl_72 100153 324955504.875583 6117899 120 0.000000 83788.953365 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.684287] ll_ost_create00 108510 323151569.415478 211668 120 0.000000 2664.761909 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.695742] ll_ost_io00_074 40229 324955297.351316 108420571 120 0.000000 7145457.845053 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.707073] ll_ost_io00_075 40230 324954829.565331 132260633 120 0.000000 8545976.114716 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.718408] ll_ost_io00_078 40233 324955069.134479 72386290 120 0.000000 5298778.179412 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.729776] ll_ost00_089 70314 315707379.009668 24358255 120 0.000000 533090.490764 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.741123] raid-check 64016 324955487.348895 561648 120 0.000000 76972.848653 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.752464] kworker/20:1 98374 324955505.297756 1541043 120 0.000000 34737.208911 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.763801] raid-check 122974 324955505.828649 27162 120 0.000000 2915.764648 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.775227] kworker/20:0 7172 324662743.829371 2 120 0.000000 0.009678 0.000000 0 / [Tue Mar 17 18:02:30 2020][8592634.786630] [Tue Mar 17 18:02:30 2020][8592634.788299] cpu#21, 1996.233 MHz [Tue Mar 17 18:02:30 2020][8592634.791707] .nr_running : 0 [Tue Mar 17 18:02:30 2020][8592634.796501] .load : 0 [Tue Mar 17 18:02:30 2020][8592634.801294] .nr_switches : 11614458169 [Tue Mar 17 18:02:30 2020][8592634.806966] .nr_load_updates : 2902907541 [Tue Mar 17 18:02:30 2020][8592634.812545] .nr_uninterruptible : -179299 [Tue Mar 17 18:02:30 2020][8592634.817865] .next_balance : 12887.279319 [Tue Mar 17 18:02:30 2020][8592634.823632] .curr->pid : 0 [Tue Mar 17 18:02:30 2020][8592634.828427] .clock : 8592634828.388398 [Tue Mar 17 18:02:30 2020][8592634.834607] .cpu_load[0] : 0 [Tue Mar 17 18:02:30 2020][8592634.839414] .cpu_load[1] : 0 [Tue Mar 17 18:02:30 2020][8592634.844211] .cpu_load[2] : 0 [Tue Mar 17 18:02:30 2020][8592634.849002] .cpu_load[3] : 0 [Tue Mar 17 18:02:30 2020][8592634.853810] .cpu_load[4] : 0 [Tue Mar 17 18:02:30 2020][8592634.858607] .avg_idle : 6442042 [Tue Mar 17 18:02:30 2020][8592634.863927] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:30 2020][8592634.869256] [Tue Mar 17 18:02:30 2020][8592634.870927] cfs_rq[21]:/ [Tue Mar 17 18:02:30 2020][8592634.873638] .exec_clock : 0.000000 [Tue Mar 17 18:02:30 2020][8592634.879038] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:30 2020][8592634.884438] .min_vruntime : 174242021.922655 [Tue Mar 17 18:02:30 2020][8592634.890530] .max_vruntime : 0.000001 [Tue Mar 17 18:02:30 2020][8592634.895930] .spread : 0.000000 [Tue Mar 17 18:02:30 2020][8592634.901330] .spread0 : -142973121.410096 [Tue Mar 17 18:02:30 2020][8592634.907509] .nr_spread_over : 0 [Tue Mar 17 18:02:30 2020][8592634.912301] .nr_running : 0 [Tue Mar 17 18:02:30 2020][8592634.917092] .load : 0 [Tue Mar 17 18:02:30 2020][8592634.921885] .runnable_load_avg : 0 [Tue Mar 17 18:02:30 2020][8592634.926680] .blocked_load_avg : 0 [Tue Mar 17 18:02:30 2020][8592634.931471] .tg_load_avg : 0 [Tue Mar 17 18:02:30 2020][8592634.936264] .tg_load_contrib : 0 [Tue Mar 17 18:02:30 2020][8592634.941058] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:30 2020][8592634.945849] .tg->runnable_avg : 0 [Tue Mar 17 18:02:30 2020][8592634.950643] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:30 2020][8592634.955436] .throttled : 0 [Tue Mar 17 18:02:30 2020][8592634.960229] .throttle_count : 0 [Tue Mar 17 18:02:30 2020][8592634.965021] .avg->runnable_avg_sum : 38 [Tue Mar 17 18:02:30 2020][8592634.969900] .avg->runnable_avg_period : 48062 [Tue Mar 17 18:02:30 2020][8592634.975042] [Tue Mar 17 18:02:30 2020][8592634.976714] rt_rq[21]:/ [Tue Mar 17 18:02:30 2020][8592634.979338] .rt_nr_running : 0 [Tue Mar 17 18:02:30 2020][8592634.984131] .rt_throttled : 0 [Tue Mar 17 18:02:30 2020][8592634.988924] .rt_time : 0.000000 [Tue Mar 17 18:02:30 2020][8592634.994322] .rt_runtime : 950.000000 [Tue Mar 17 18:02:30 2020][8592634.999896] [Tue Mar 17 18:02:30 2020][8592635.001568] runnable tasks: [Tue Mar 17 18:02:30 2020][8592635.004541] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:30 2020][8592635.015488] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:30 2020][8592635.026367] watchdog/21 116 -11.980063 2148156 0 0.000000 20656.830476 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.037700] migration/21 117 0.000000 72571 0 0.000000 73154.209253 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.049035] ksoftirqd/21 118 174241799.524713 457532787 120 0.000000 8876613.214677 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.060373] kworker/21:0H 120 3209.195985 10 100 0.000000 0.053370 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.071733] kswapd1 277 174241332.759076 2713149 120 0.000000 13961152.839284 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.083114] kdmwork-253:13 12611 948604.010915 226726 120 0.000000 3195.969965 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.094465] kworker/21:1H 14274 174241799.517148 920019443 100 0.000000 6250258.527605 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.105859] kiblnd_sd_01_03 64789 174242022.107052 6693240674 120 0.000000 50976313.567081 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.117284] ptlrpcd_01_01 64818 174242001.692299 4533373 120 0.000000 136308.762997 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.128649] ll_ost01_008 68323 168601101.298093 21059666 120 0.000000 504865.636923 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.139991] ll_ost01_011 68367 174241584.384945 22235528 120 0.000000 597045.119453 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.151329] ll_ost_io01_008 68372 174241803.057356 43509722 120 0.000000 3120279.987804 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.162669] ll_ost_io01_012 68396 174241811.516638 22421687 120 0.000000 1693530.033485 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.174019] ll_ost01_030 68587 152589994.755006 15275300 120 0.000000 369538.190353 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.185368] ll_ost_io01_021 68718 174241802.691838 100176669 120 0.000000 7378989.222324 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.196711] ll_ost01_074 68800 168601101.327498 17609803 120 0.000000 470355.506678 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.208044] ll_ost01_076 68810 174241355.525594 23390875 120 0.000000 603336.736423 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.219380] ll_ost_io01_029 68817 174241803.443635 46510481 120 0.000000 3435281.399763 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.230719] ll_ost_io01_033 68845 174241800.841062 70715257 120 0.000000 5067108.899107 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.242056] ll_ost_io01_044 68867 174241705.336691 51658663 120 0.000000 3651151.922541 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.253390] ll_ost_io01_047 68875 174241633.123669 34213420 120 0.000000 2427052.938041 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.264727] ll_ost_io01_053 68892 174241624.338026 23976444 120 0.000000 1863009.175574 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.276070] ll_ost_io01_065 73696 174241626.322904 52965183 120 0.000000 3317567.473464 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.287408] ll_ost_io01_076 73707 174241249.726764 93782057 120 0.000000 6683270.111951 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.298746] ll_ost_io01_084 73715 174241680.482833 60585784 120 0.000000 4258601.975018 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.310080] ll_ost_io01_091 73722 174241800.172032 58185693 120 0.000000 3670368.863116 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.321427] ldlm_bl_37 100118 174242007.518761 6105715 120 0.000000 84166.941369 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.332853] ldlm_bl_65 100146 174242007.861460 6114631 120 0.000000 84185.258389 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.344283] ldlm_bl_91 100172 174242008.294475 6118258 120 0.000000 84045.147436 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.355746] ll_ost_io01_093 40245 174241274.597511 74314671 120 0.000000 4939042.293751 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.367082] ll_ost_io01_097 40786 174241802.137143 93618870 120 0.000000 6850455.578343 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.378416] ll_ost_io01_098 40789 174241533.808274 66449100 120 0.000000 4478072.944424 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.389756] ll_ost_io01_109 40826 174241799.547736 74413124 120 0.000000 4925949.919367 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.401098] ll_ost01_104 101314 144932765.325123 13451159 120 0.000000 358104.906174 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.412545] raid-check 24351 174242001.431108 96978 120 0.000000 11653.878440 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.423883] kworker/21:1 102099 174242009.529545 606368 120 0.000000 11827.839554 0.000000 1 / [Tue Mar 17 18:02:30 2020][8592635.435315] kworker/21:0 11080 174103111.449608 2 120 0.000000 0.009809 0.000000 1 / [Tue Mar 17 18:02:31 2020][8592635.446650] pickup 13120 174242004.847095 77 120 0.000000 11.848366 0.000000 1 / [Tue Mar 17 18:02:31 2020][8592635.457988] trivial-rewrite 14290 174242012.422540 9 120 0.000000 5.064079 0.000000 1 / [Tue Mar 17 18:02:31 2020][8592635.469331] [Tue Mar 17 18:02:31 2020][8592635.471004] cpu#22, 1996.233 MHz [Tue Mar 17 18:02:31 2020][8592635.474411] .nr_running : 1 [Tue Mar 17 18:02:31 2020][8592635.479205] .load : 1024 [Tue Mar 17 18:02:31 2020][8592635.484257] .nr_switches : 17808479983 [Tue Mar 17 18:02:31 2020][8592635.489916] .nr_load_updates : 3763263050 [Tue Mar 17 18:02:31 2020][8592635.495504] .nr_uninterruptible : 13808130 [Tue Mar 17 18:02:31 2020][8592635.500905] .next_balance : 12887.280003 [Tue Mar 17 18:02:31 2020][8592635.506651] .curr->pid : 0 [Tue Mar 17 18:02:31 2020][8592635.511459] .clock : 8592635511.444965 [Tue Mar 17 18:02:31 2020][8592635.517641] .cpu_load[0] : 0 [Tue Mar 17 18:02:31 2020][8592635.522433] .cpu_load[1] : 0 [Tue Mar 17 18:02:31 2020][8592635.527228] .cpu_load[2] : 0 [Tue Mar 17 18:02:31 2020][8592635.532047] .cpu_load[3] : 0 [Tue Mar 17 18:02:31 2020][8592635.536840] .cpu_load[4] : 0 [Tue Mar 17 18:02:31 2020][8592635.541639] .avg_idle : 5760574 [Tue Mar 17 18:02:31 2020][8592635.546967] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:31 2020][8592635.552295] [Tue Mar 17 18:02:31 2020][8592635.553969] cfs_rq[22]:/ [Tue Mar 17 18:02:31 2020][8592635.556682] .exec_clock : 0.000000 [Tue Mar 17 18:02:31 2020][8592635.562080] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:31 2020][8592635.567481] .min_vruntime : 208874643.961248 [Tue Mar 17 18:02:31 2020][8592635.573572] .max_vruntime : 0.000001 [Tue Mar 17 18:02:31 2020][8592635.578973] .spread : 0.000000 [Tue Mar 17 18:02:31 2020][8592635.584371] .spread0 : -108340499.371503 [Tue Mar 17 18:02:31 2020][8592635.590551] .nr_spread_over : 0 [Tue Mar 17 18:02:31 2020][8592635.595344] .nr_running : 0 [Tue Mar 17 18:02:31 2020][8592635.600136] .load : 0 [Tue Mar 17 18:02:31 2020][8592635.604927] .runnable_load_avg : 0 [Tue Mar 17 18:02:31 2020][8592635.609721] .blocked_load_avg : 0 [Tue Mar 17 18:02:31 2020][8592635.614515] .tg_load_avg : 0 [Tue Mar 17 18:02:31 2020][8592635.619306] .tg_load_contrib : 0 [Tue Mar 17 18:02:31 2020][8592635.624099] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:31 2020][8592635.628892] .tg->runnable_avg : 0 [Tue Mar 17 18:02:31 2020][8592635.633684] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:31 2020][8592635.638478] .throttled : 0 [Tue Mar 17 18:02:31 2020][8592635.643270] .throttle_count : 0 [Tue Mar 17 18:02:31 2020][8592635.648062] .avg->runnable_avg_sum : 81 [Tue Mar 17 18:02:31 2020][8592635.652943] .avg->runnable_avg_period : 47271 [Tue Mar 17 18:02:31 2020][8592635.658083] [Tue Mar 17 18:02:31 2020][8592635.659755] rt_rq[22]:/ [Tue Mar 17 18:02:31 2020][8592635.662381] .rt_nr_running : 0 [Tue Mar 17 18:02:31 2020][8592635.667174] .rt_throttled : 0 [Tue Mar 17 18:02:31 2020][8592635.671965] .rt_time : 0.008286 [Tue Mar 17 18:02:31 2020][8592635.677366] .rt_runtime : 950.000000 [Tue Mar 17 18:02:31 2020][8592635.682938] [Tue Mar 17 18:02:31 2020][8592635.684610] runnable tasks: [Tue Mar 17 18:02:31 2020][8592635.687584] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:31 2020][8592635.698528] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:31 2020][8592635.709406] watchdog/22 121 -11.980533 2148157 0 0.000000 22827.382136 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.720741] migration/22 122 0.000000 122763 0 0.000000 212564.831478 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.732079] ksoftirqd/22 123 208874495.187253 481874518 120 0.000000 6523810.363708 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.743415] kworker/22:0H 125 121502411.796576 11 100 0.000000 0.086561 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.754782] kworker/22:1H 5595 208873419.544894 1408665962 100 0.000000 7844099.389368 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.766200] systemd-journal 5729 208874643.961248 900233 120 0.000000 229906.522475 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.777534] systemd-udevd 5754 639833.124549 4823 120 0.000000 10338.550414 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.788925] kdmwork-253:45 18014 208872383.550475 879095211 120 0.000000 24806997.589106 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.800292] kdmwork-253:114 26325 208872376.924296 830217604 120 0.000000 29490312.485478 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.811636] dbus-daemon 50204 208874554.443546 6359661 120 0.000000 577140.579665 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.822966] chronyd 50237 208873753.958167 230331 120 0.000000 19497.716075 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.834304] sshd 50881 208871747.491826 1121 120 0.000000 190.362261 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.845638] writer#1 50895 208874616.047075 48838385 120 0.000000 723254.086095 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.856994] ptlrpcd_02_02 64831 208872019.656534 2460322 120 0.000000 101880.018064 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.868328] ptlrpcd_02_03 64832 208872019.579930 2557297 120 0.000000 99473.884608 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.879671] md8_raid6 66071 208872004.101256 1493225604 120 0.000000 462331162.988254 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.891102] ldlm_cn02_000 66889 208871738.787648 23792017 120 0.000000 579904.592674 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.902444] ll_ost_io02_000 66955 208872118.174256 40819216 120 0.000000 2516407.376633 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.913787] lquota_wb_fir-O 68253 208874563.376630 175258 120 0.000000 3604.231824 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.925139] ll_ost02_020 68487 208872047.722570 45980948 120 0.000000 1298225.520025 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.936477] ll_ost_io02_018 68503 208872045.261043 189591298 120 0.000000 12318197.701165 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.947812] ll_ost_io02_020 68510 208872296.554143 149595422 120 0.000000 10144418.964112 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.959157] ll_ost02_034 68598 208872141.764208 38188529 120 0.000000 1099678.964359 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.970493] ll_ost02_037 68611 204771576.080823 41545906 120 0.000000 1184843.533115 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.981828] ll_ost02_040 68628 208872272.359489 42644171 120 0.000000 1198224.550499 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592635.993163] ll_ost02_041 68630 208872141.789427 36616761 120 0.000000 1011597.417766 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.004507] ll_ost02_057 68710 208872377.707081 50305831 120 0.000000 1385415.804989 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.015853] ll_ost_io02_032 68790 208872029.060908 101260948 120 0.000000 6706772.051988 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.027191] ll_ost02_075 68797 208872362.097589 43771287 120 0.000000 1263455.740001 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.038528] ll_ost_io02_039 68841 208872072.246345 83929598 120 0.000000 5804362.648833 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.049861] ll_ost_io02_040 68843 208872107.477414 37862009 120 0.000000 2322867.573422 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.061204] ll_ost_create02 69205 207224628.975872 4236675 120 0.000000 82439.010639 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.072542] ldlm_cn02_006 69427 208873361.849695 30859675 120 0.000000 717046.569331 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.083894] ll_ost02_079 98670 208872215.299658 42959696 120 0.000000 1264329.704702 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.095232] ll_ost02_085 98676 208872376.980684 44458124 120 0.000000 1287081.628710 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.106569] ll_ost02_094 98685 208872362.006106 43183490 120 0.000000 1274150.924569 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.117921] ldlm_cb02_002 102536 208182423.347530 2092 120 0.000000 39.381132 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.129345] ldlm_cn02_010 111298 208871739.394872 32150236 120 0.000000 774929.002111 0.000000 2 /[8592636.140408] LNet: Service thread pid 67233 was inactive for 200.68s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. [Tue Mar 17 18:02:31 2020][8592636.140410] LNet: Skipped 35 previous similar messages [Tue Mar 17 18:02:31 2020][8592636.140412] LustreError: dumping log to /tmp/lustre-log.1584493351.67233 [Tue Mar 17 18:02:31 2020] [Tue Mar 17 18:02:31 2020][8592636.165992] ldlm_cn02_020 31076 208871674.797207 12335593 120 0.000000 299160.565599 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.177322] ldlm_cn02_024 64117 208874629.352855 17283474 120 0.000000 410847.067991 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.188673] ll_ost_io02_072 90841 208872212.830034 125713656 120 0.000000 7871691.475135 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.200021] ll_ost_io02_098 40787 208872088.762124 35882465 120 0.000000 2486452.247081 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.211362] ll_ost_io02_106 40815 208872205.439757 85423071 120 0.000000 4992478.123793 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.222726] python 128778 208874632.623165 20970681 120 0.000000 46853003.692037 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.234161] kworker/22:2 3537 208695376.936677 642 120 0.000000 1.269900 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.245499] kworker/22:1 6330 208874629.351553 45153 120 0.000000 866.144279 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.256835] bash 11738 208764572.889883 87 120 0.000000 13.682399 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.268173] sshd 14039 208871994.072344 98 120 0.000000 20.366921 0.000000 2 / [Tue Mar 17 18:02:31 2020][8592636.279545] [Tue Mar 17 18:02:31 2020][8592636.281222] cpu#23, 1996.233 MHz [Tue Mar 17 18:02:31 2020][8592636.284629] .nr_running : 0 [Tue Mar 17 18:02:31 2020][8592636.289421] .load : 0 [Tue Mar 17 18:02:31 2020][8592636.294217] .nr_switches : 13659953922 [Tue Mar 17 18:02:31 2020][8592636.299910] .nr_load_updates : 3281917917 [Tue Mar 17 18:02:31 2020][8592636.305482] .nr_uninterruptible : -175520 [Tue Mar 17 18:02:31 2020][8592636.310795] .next_balance : 12887.280811 [Tue Mar 17 18:02:31 2020][8592636.316555] .curr->pid : 0 [Tue Mar 17 18:02:31 2020][8592636.321351] .clock : 8592636320.392608 [Tue Mar 17 18:02:31 2020][8592636.327530] .cpu_load[0] : 0 [Tue Mar 17 18:02:31 2020][8592636.332337] .cpu_load[1] : 0 [Tue Mar 17 18:02:31 2020][8592636.337132] .cpu_load[2] : 0 [Tue Mar 17 18:02:31 2020][8592636.341924] .cpu_load[3] : 0 [Tue Mar 17 18:02:31 2020][8592636.346733] .cpu_load[4] : 0 [Tue Mar 17 18:02:31 2020][8592636.351541] .avg_idle : 10000000 [Tue Mar 17 18:02:31 2020][8592636.356944] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:31 2020][8592636.362271] [Tue Mar 17 18:02:31 2020][8592636.363943] cfs_rq[23]:/ [Tue Mar 17 18:02:31 2020][8592636.366655] .exec_clock : 0.000000 [Tue Mar 17 18:02:31 2020][8592636.372056] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:31 2020][8592636.377454] .min_vruntime : 179305706.334374 [Tue Mar 17 18:02:31 2020][8592636.383549] .max_vruntime : 0.000001 [Tue Mar 17 18:02:31 2020][8592636.388948] .spread : 0.000000 [Tue Mar 17 18:02:31 2020][8592636.394345] .spread0 : -137909436.998377 [Tue Mar 17 18:02:31 2020][8592636.400525] .nr_spread_over : 0 [Tue Mar 17 18:02:31 2020][8592636.405318] .nr_running : 0 [Tue Mar 17 18:02:31 2020][8592636.410110] .load : 0 [Tue Mar 17 18:02:31 2020][8592636.414904] .runnable_load_avg : 0 [Tue Mar 17 18:02:31 2020][8592636.419697] .blocked_load_avg : 0 [Tue Mar 17 18:02:32 2020][8592636.424488] .tg_load_avg : 0 [Tue Mar 17 18:02:32 2020][8592636.429281] .tg_load_contrib : 0 [Tue Mar 17 18:02:32 2020][8592636.434073] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:32 2020][8592636.438866] .tg->runnable_avg : 0 [Tue Mar 17 18:02:32 2020][8592636.443660] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:32 2020][8592636.448453] .throttled : 0 [Tue Mar 17 18:02:32 2020][8592636.453246] .throttle_count : 0 [Tue Mar 17 18:02:32 2020][8592636.458039] .avg->runnable_avg_sum : 4 [Tue Mar 17 18:02:32 2020][8592636.462830] .avg->runnable_avg_period : 47288 [Tue Mar 17 18:02:32 2020][8592636.467972] [Tue Mar 17 18:02:32 2020][8592636.469641] rt_rq[23]:/ [Tue Mar 17 18:02:32 2020][8592636.472270] .rt_nr_running : 0 [Tue Mar 17 18:02:32 2020][8592636.477062] .rt_throttled : 0 [Tue Mar 17 18:02:32 2020][8592636.481853] .rt_time : 0.000000 [Tue Mar 17 18:02:32 2020][8592636.487255] .rt_runtime : 950.000000 [Tue Mar 17 18:02:32 2020][8592636.492826] [Tue Mar 17 18:02:32 2020][8592636.494499] runnable tasks: [Tue Mar 17 18:02:32 2020][8592636.497473] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:32 2020][8592636.508418] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:32 2020][8592636.519299] watchdog/23 126 -11.980052 2148159 0 0.000000 17840.585840 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.530631] migration/23 127 0.000000 70853 0 0.000000 91774.166879 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.541965] ksoftirqd/23 128 179305688.611574 548201468 120 0.000000 9357747.542161 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.553303] kworker/23:0H 130 1152.675653 10 100 0.000000 0.040155 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.564668] kworker/23:1H 5594 179305688.592531 1030657575 100 0.000000 6374928.215454 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.576098] multipathd 11205 11910.008780 1172 120 0.000000 34.398948 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.587432] multipathd 17891 0.000000 4 0 0.000000 1.571168 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.598769] multipathd 17895 0.000000 7 0 0.000000 0.927899 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.610104] multipathd 17899 0.000000 4 0 0.000000 0.956771 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.621441] multipathd 17903 0.000000 3 0 0.000000 0.946624 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.632775] multipathd 17905 0.000000 4 0 0.000000 0.961120 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.644114] multipathd 17910 0.000000 3 0 0.000000 0.937866 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.655450] multipathd 17912 0.000000 4 0 0.000000 1.012306 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.666785] multipathd 17917 0.000000 6 0 0.000000 0.974326 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.678121] multipathd 17921 0.000000 5 0 0.000000 0.854659 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.689457] multipathd 17926 0.000000 4 0 0.000000 1.070426 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.700794] multipathd 17930 0.000000 3 0 0.000000 1.133695 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.712129] multipathd 17932 0.000000 4 0 0.000000 1.386512 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.723465] multipathd 17934 0.000000 5 0 0.000000 1.107726 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.734801] multipathd 17936 0.000000 6 0 0.000000 1.134306 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.746137] multipathd 17938 0.000000 3 0 0.000000 1.443149 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.757474] multipathd 17941 0.000000 4 0 0.000000 1.195742 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.768811] multipathd 17943 0.000000 3 0 0.000000 1.138945 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.780146] multipathd 17945 0.000000 3 0 0.000000 1.143463 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.791481] multipathd 17947 0.000000 4 0 0.000000 1.229177 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.802835] kdmflush 12600 3325.367677 2 100 0.000000 0.012724 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.814170] bioset 12601 3325.371581 2 100 0.000000 0.012284 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.825508] kdmflush 12604 3325.383249 2 100 0.000000 0.013485 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.836842] bioset 12605 3325.387031 2 100 0.000000 0.012474 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.848180] kdmflush 12607 3325.408057 2 100 0.000000 0.014728 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.859517] bioset 12610 3325.412019 2 100 0.000000 0.013154 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.870866] kdmwork-253:37 12731 179305595.828223 822701122 120 0.000000 30150912.649459 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.882206] kdmflush 18018 3535.433927 2 100 0.000000 0.010109 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.893540] bioset 18019 3535.435785 2 100 0.000000 0.010740 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.904879] kdmflush 18028 3535.448409 2 100 0.000000 0.011863 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.916213] bioset 18029 3535.450132 2 100 0.000000 0.009548 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.927548] kdmflush 18033 3535.460330 2 100 0.000000 0.011451 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.938885] bioset 18034 3535.462252 2 100 0.000000 0.010058 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.950223] kdmflush 18060 3535.475008 2 100 0.000000 0.012153 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.961556] bioset 18061 3535.476760 2 100 0.000000 0.010340 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.972905] kdmwork-253:72 18401 179305598.263173 812757798 120 0.000000 28313757.856130 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.984239] kdmflush 18416 3535.501604 2 100 0.000000 0.013004 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592636.995574] bioset 18417 3535.503216 2 100 0.000000 0.008526 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.006918] bioset 19070 3549.067832 2 100 0.000000 0.010459 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.018260] kdmwork-253:102 19664 179305586.274735 828126987 120 0.000000 29034730.933143 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.029592] kdmflush 20036 3562.750662 2 100 0.000000 0.011322 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.040927] bioset 20038 3562.751873 2 100 0.000000 0.007414 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.052265] kdmflush 21023 3585.605437 2 100 0.000000 0.012715 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.063600] bioset 24527 3895.932965 2 100 0.000000 0.009488 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.074935] kdmflush 25205 4375.337256 2 100 0.000000 0.009508 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.086271] bioset 25206 4375.339001 2 100 0.000000 0.007684 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.097610] kdmflush 25232 4395.845511 2 100 0.000000 0.026830 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.108945] bioset 25246 4406.577596 2 100 0.000000 0.009889 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.120279] kdmflush 26032 4724.356744 2 100 0.000000 0.010760 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.131616] bioset 26324 4853.877224 2 100 0.000000 0.014447 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.142951] kdmflush 27524 5472.515474 2 100 0.000000 0.014067 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.154287] kdmflush 27772 5593.911852 2 100 0.000000 0.011852 0.000000[8592637.164413] LustreError: dumping log to /tmp/lustre-log.1584493352.68542 [Tue Mar 17 18:02:32 2020] 3 / [Tue Mar 17 18:02:32 2020][8592637.172496] bioset 27773 5593.913885 2 100 0.000000 0.009177 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.183834] bioset 27831 5630.682791 2 100 0.000000 0.012714 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.195192] ptlrpcd_03_06 64847 179304394.282753 3446654 120 0.000000 119703.373076 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.206535] ldlm_bl_02 66896 179305693.013592 6334376 120 0.000000 87617.668535 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.217868] ll_cfg_requeue 66899 38533862.837657 450 120 0.000000 19.601843 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.229205] ll_ost_create03 66939 179305694.223552 1663924 120 0.000000 28104.577907 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.240548] lquota_wb_fir-O 68171 179305609.165623 178504 120 0.000000 3344.957933 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.251890] ll_ost_io03_003 68351 179305379.305556 79428721 120 0.000000 5170681.077790 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.263223] ll_ost_io03_006 68379 179305589.290097 158234832 120 0.000000 10131777.406845 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.274558] ll_ost_io03_007 68384 179305407.160815 67644914 120 0.000000 4277255.126472 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.285896] ldlm_bl_03 68399 179305691.853592 6334820 120 0.000000 87125.316096 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.297234] ll_ost_io03_013 68486 179305525.815817 84790078 120 0.000000 5304742.629555 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.308573] ll_ost03_017 68541 178717775.480626 14508320 120 0.000000 446488.856539 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.319914] ll_ost03_024 68563 179305587.086891 24781769 120 0.000000 675923.226720 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.331245] ll_ost03_026 68569 179305607.054873 17576048 120 0.000000 534830.096970 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.342585] ll_ost03_029 68582 170206076.966011 19752693 120 0.000000 604367.956862 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.353929] ll_ost03_056 68678 170206076.965710 18639408 120 0.000000 554820.162579 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.365264] ll_ost03_060 68715 179305612.749586 13951711 120 0.000000 434024.187942 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.376604] ll_ost_io03_026 68773 179304961.119362 123751095 120 0.000000 7928417.571757 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.387939] ll_ost_io03_030 68816 179305065.789417 77075863 120 0.000000 4558453.065211 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.399277] ll_ost_io03_036 68876 179305378.397294 61513282 120 0.000000 4121158.967319 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.410616] ll_ost_io03_039 68883 179305532.653449 61742436 120 0.000000 3998357.631987 0.000000 3 / [Tue Mar 17 18:02:32 2020][8592637.421974] ll_ost_io03_047 97771 179305035.339974 96518843 120 0.000000 6215579.720316 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.433310] ldlm_bl_23 100104 179305692.056001 6108933 120 0.000000 149475.860230 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.444729] ldlm_bl_27 100108 179305695.047732 6112999 120 0.000000 83987.948254 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.456149] ldlm_bl_28 100109 179305692.126823 6111703 120 0.000000 83985.989874 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.467575] ldlm_bl_36 100117 179305692.621564 6103804 120 0.000000 83223.638377 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.478996] ldlm_bl_40 100121 179305693.083625 6108378 120 0.000000 83734.509724 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.490420] ldlm_bl_45 100126 179305694.145905 6109081 120 0.000000 83448.661566 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.501847] ldlm_bl_70 100151 179305694.668906 6110833 120 0.000000 83655.905299 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.513273] ldlm_bl_71 100152 179305694.781528 6111553 120 0.000000 83622.699358 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.524699] ldlm_bl_80 100161 179305694.366606 6110701 120 0.000000 83980.834595 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.536118] ldlm_bl_89 100170 179305691.861156 6116096 120 0.000000 84060.961975 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.547542] ldlm_bl_94 100175 179305692.075499 6105476 120 0.000000 83635.079526 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.558984] ldlm_cn03_031 64137 178892033.415075 8560043 120 0.000000 195189.347395 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.570319] ll_ost_io03_053 87185 179305583.821400 137475444 120 0.000000 8402066.373431 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.581661] ll_ost_io03_065 90847 179305586.314280 106568216 120 0.000000 6443581.939901 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.593017] ll_ost_io03_089 40818 179305455.035792 57544633 120 0.000000 3598708.466214 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.604353] ll_ost_io03_090 40822 179304999.743645 85740887 120 0.000000 4424219.021854 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.615694] ll_ost_io03_098 40844 179304404.045680 102478301 120 0.000000 6568014.513480 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.627049] ll_ost03_078 70325 179305707.600656 6530113 120 0.000000 190933.975404 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.638388] ll_ost03_091 71289 179305620.126813 18693420 120 0.000000 524747.248489 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.649729] ll_ost03_100 90275 170206076.965489 6045657 120 0.000000 210860.829776 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.661067] ll_ost03_106 125739 178717775.481096 17547423 120 0.000000 478370.312948 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.672494] md8_resync 55908 179304253.608865 393288985 125 0.000000 17884763.986373 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.683826] kworker/23:2 106607 179305691.840138 2523250 120 0.000000 44340.678424 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.695247] md10_resync 24401 179304962.224394 222634754 125 0.000000 10205678.869332 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.706593] kworker/23:1 1238 178837236.243247 2 120 0.000000 0.008937 0.000000 3 / [Tue Mar 17 18:02:33 2020][8592637.717985] [Tue Mar 17 18:02:33 2020][8592637.719657] cpu#24, 1996.233 MHz [Tue Mar 17 18:02:33 2020][8592637.723063] .nr_running : 0 [Tue Mar 17 18:02:33 2020][8592637.727856] .load : 0 [Tue Mar 17 18:02:33 2020][8592637.732648] .nr_switches : 15910113523 [Tue Mar 17 18:02:33 2020][8592637.738324] .nr_load_updates : 3040614650 [Tue Mar 17 18:02:33 2020][8592637.743897] .nr_uninterruptible : 577815 [Tue Mar 17 18:02:33 2020][8592637.749126] .next_balance : 12887.282249 [Tue Mar 17 18:02:33 2020][8592637.754893] .curr->pid : 0 [Tue Mar 17 18:02:33 2020][8592637.759691] .clock : 8592637759.396562 [Tue Mar 17 18:02:33 2020][8592637.765878] .cpu_load[0] : 0 [Tue Mar 17 18:02:33 2020][8592637.770685] .cpu_load[1] : 0 [Tue Mar 17 18:02:33 2020][8592637.775480] .cpu_load[2] : 0 [Tue Mar 17 18:02:33 2020][8592637.780273] .cpu_load[3] : 0 [Tue Mar 17 18:02:33 2020][8592637.785066] .cpu_load[4] : 0 [Tue Mar 17 18:02:33 2020][8592637.789873] .avg_idle : 5772581 [Tue Mar 17 18:02:33 2020][8592637.795189] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:33 2020][8592637.800499] [Tue Mar 17 18:02:33 2020][8592637.802169] cfs_rq[24]:/ [Tue Mar 17 18:02:33 2020][8592637.804882] .exec_clock : 0.000000 [Tue Mar 17 18:02:33 2020][8592637.810282] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:33 2020][8592637.815680] .min_vruntime : 156301681.071283 [Tue Mar 17 18:02:33 2020][8592637.821775] .max_vruntime : 0.000001 [Tue Mar 17 18:02:33 2020][8592637.827172] .spread : 0.000000 [Tue Mar 17 18:02:33 2020][8592637.832571] .spread0 : -160913462.261468 [Tue Mar 17 18:02:33 2020][8592637.838751] .nr_spread_over : 0 [Tue Mar 17 18:02:33 2020][8592637.843544] .nr_running : 0 [Tue Mar 17 18:02:33 2020][8592637.848336] .load : 0 [Tue Mar 17 18:02:33 2020][8592637.853129] .runnable_load_avg : 0 [Tue Mar 17 18:02:33 2020][8592637.857923] .blocked_load_avg : 0 [Tue Mar 17 18:02:33 2020][8592637.862714] .tg_load_avg : 0 [Tue Mar 17 18:02:33 2020][8592637.867508] .tg_load_contrib : 0 [Tue Mar 17 18:02:33 2020][8592637.872302] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:33 2020][8592637.877092] .tg->runnable_avg : 0 [Tue Mar 17 18:02:33 2020][8592637.881886] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:33 2020][8592637.886680] .throttled : 0 [Tue Mar 17 18:02:33 2020][8592637.891470] .throttle_count : 0 [Tue Mar 17 18:02:33 2020][8592637.896264] .avg->runnable_avg_sum : 14 [Tue Mar 17 18:02:33 2020][8592637.901143] .avg->runnable_avg_period : 46958 [Tue Mar 17 18:02:33 2020][8592637.906285] [Tue Mar 17 18:02:33 2020][8592637.907955] rt_rq[24]:/ [Tue Mar 17 18:02:33 2020][8592637.910581] .rt_nr_running : 0 [Tue Mar 17 18:02:33 2020][8592637.915375] .rt_throttled : 0 [Tue Mar 17 18:02:33 2020][8592637.920168] .rt_time : 0.000000 [Tue Mar 17 18:02:33 2020][8592637.925566] .rt_runtime : 950.000000 [Tue Mar 17 18:02:33 2020][8592637.931137] [Tue Mar 17 18:02:33 2020][8592637.932812] runnable tasks: [Tue Mar 17 18:02:33 2020][8592637.935783] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:33 2020][8592637.946730] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:33 2020][8592637.957612] watchdog/24 131 -11.981465 2148156 0 0.000000 16130.728009 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592637.968943] migration/24 132 0.000000 58057 0 0.000000 31345.647543 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592637.980279] ksoftirqd/24 133 156301595.899102 169433399 120 0.000000 1936584.366466 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592637.991614] kworker/24:0H 135 7134.904805 10 100 0.000000 0.056818 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.003024] kdmwork-253:28 12703 156301606.689837 878557297 120 0.000000 30838346.808999 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.014377] kworker/24:1H 18068 156301595.912976 244053985 100 0.000000 1042678.883168 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.025742] kdmflush 26321 12165.370260 2 100 0.000000 0.014478 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.037080] gssproxy 50217 25994.689224 1 120 0.000000 0.031198 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.048432] ptlrpcd_00_08 64813 156301247.552573 4145283 120 0.000000 143928.230280 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.059790] ll_ost_io00_001 66950 156301329.778162 110220127 120 0.000000 7883314.955947 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.071136] lquota_wb_fir-O 68307 156301655.855553 175575 120 0.000000 3588.907917 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.082475] ll_ost_io00_005 68364 156301193.381053 185700197 120 0.000000 11632189.703838 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.093815] ll_ost_io00_006 68371 156301264.633222 156799557 120 0.000000 10248451.551486 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.105169] ll_ost00_017 68556 151767228.113059 20860083 120 0.000000 479188.932825 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.116511] ll_ost00_025 68586 156301611.165069 50789493 120 0.000000 1131811.984454 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.127867] ll_ost_io00_029 68770 156301251.314131 99308737 120 0.000000 6649577.710662 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.139208] ll_ost00_068 68806 154423458.602573 70185770 120 0.000000 1508252.798054 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.150547] ll_ost00_075 68820 153904593.532323 76082899 120 0.000000 1643604.888667 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.161884] ll_ost_io00_034 68833 156301583.183928 130521590 120 0.000000 8753824.312816 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.173219] ll_ost_io00_037 68844 156301286.299551 43366955 120 0.000000 3327000.641660 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.184556] ll_ost_io00_039 68858 156301224.285483 119933899 120 0.000000 7316425.852645 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.195892] ll_ost_io00_043 68882 156301595.952391 146097519 120 0.000000 9621467.566983 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.207231] ll_ost_io00_052 68902 156301234.787950 65853397 120 0.000000 4528609.703441 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.218591] ldlm_bl_48 100129 156301666.889639 6115870 120 0.000000 83937.542959 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.230015] ldlm_bl_63 100144 156301666.892517 6110462 120 0.000000 83946.611924 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.241438] ldlm_bl_77 100158 156301666.879421 6113479 120 0.000000 83621.906821 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.252860] ldlm_bl_95 100176 156301666.881575 6107522 120 0.000000 83585.905070 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.264310] ll_ost_io00_065 90854 156301261.336443 56148071 120 0.000000 3576585.373573 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.275649] ll_ost_io00_076 40231 156301578.057844 92935478 120 0.000000 7022748.840767 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.286989] ll_ost_io00_086 40241 156301244.356328 70736377 120 0.000000 4632554.214912 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.298338] ll_ost_io00_101 40833 156301231.770595 130072050 120 0.000000 8392931.085155 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.309678] ll_ost_io00_104 40843 156301559.187851 133821795 120 0.000000 8369070.229258 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.321014] ll_ost_io00_107 40850 156301436.889771 35545062 120 0.000000 2579622.944531 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.332349] ll_ost_io00_110 40855 156301555.005591 95436962 120 0.000000 6391120.081794 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.343689] ll_ost00_079 101316 153964332.720973 63027216 120 0.000000 1341495.487584 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.355111] ll_ost00_083 101328 156301681.419035 50153735 120 0.000000 1084793.708979 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.366532] ll_ost00_085 101334 151767228.194022 42751529 120 0.000000 957768.271609 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.377952] ll_ost00_086 101335 151767228.240438 27871020 120 0.000000 609838.110456 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.389388] ll_ost00_105 85274 151767228.100555 20391039 120 0.000000 438850.912149 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.400719] ll_ost00_107 85276 154045591.756774 30365235 120 0.000000 669454.428064 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.412056] ll_ost00_109 125734 151767228.320379 38281158 120 0.000000 847700.636139 0.000000 0 / [Tue Mar 17 18:02:33 2020][8592638.423482] ldlm_cb00_002 6628 128554722.698911 10 120 0.000000 0.204806 0.000000 0 / [Tue Mar 17 18:02:34 2020][8592638.434823] kworker/24:0 64903 155967852.656774 713 120 0.000000 2.513561 0.000000 0 / [Tue Mar 17 18:02:34 2020][8592638.446166] kworker/24:1 4645 156301669.606637 22832 120 0.000000 330.891942 0.000000 0 / [Tue Mar 17 18:02:34 2020][8592638.457564] [Tue Mar 17 18:02:34 2020][8592638.459240] cpu#25, 1996.233 MHz [Tue Mar 17 18:02:34 2020][8592638.462649] .nr_running : 0 [Tue Mar 17 18:02:34 2020][8592638.467448] .load : 0 [Tue Mar 17 18:02:34 2020][8592638.472242] .nr_switches : 8488701982 [Tue Mar 17 18:02:34 2020][8592638.477829] .nr_load_updates : 2289839197 [Tue Mar 17 18:02:34 2020][8592638.483406] .nr_uninterruptible : 201558 [Tue Mar 17 18:02:34 2020][8592638.488641] .next_balance : 12887.282989 [Tue Mar 17 18:02:34 2020][8592638.494411] .curr->pid : 0 [Tue Mar 17 18:02:34 2020][8592638.499204] .clock : 8592638498.398742 [Tue Mar 17 18:02:34 2020][8592638.505383] .cpu_load[0] : 0 [Tue Mar 17 18:02:34 2020][8592638.510190] .cpu_load[1] : 0 [Tue Mar 17 18:02:34 2020][8592638.514987] .cpu_load[2] : 0 [Tue Mar 17 18:02:34 2020][8592638.519786] .cpu_load[3] : 0 [Tue Mar 17 18:02:34 2020][8592638.524596] .cpu_load[4] : 0 [Tue Mar 17 18:02:34 2020][8592638.529392] .avg_idle : 10000000 [Tue Mar 17 18:02:34 2020][8592638.534813] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:34 2020][8592638.540155] [Tue Mar 17 18:02:34 2020][8592638.541823] cfs_rq[25]:/ [Tue Mar 17 18:02:34 2020][8592638.544534] .exec_clock : 0.000000 [Tue Mar 17 18:02:34 2020][8592638.549936] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:34 2020][8592638.555335] .min_vruntime : 81484942.258728 [Tue Mar 17 18:02:34 2020][8592638.561340] .max_vruntime : 0.000001 [Tue Mar 17 18:02:34 2020][8592638.566740] .spread : 0.000000 [Tue Mar 17 18:02:34 2020][8592638.572139] .spread0 : -235730201.074023 [Tue Mar 17 18:02:34 2020][8592638.578317] .nr_spread_over : 0 [Tue Mar 17 18:02:34 2020][8592638.583110] .nr_running : 0 [Tue Mar 17 18:02:34 2020][8592638.587904] .load : 0 [Tue Mar 17 18:02:34 2020][8592638.592698] .runnable_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592638.597488] .blocked_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592638.602283] .tg_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592638.607076] .tg_load_contrib : 0 [Tue Mar 17 18:02:34 2020][8592638.611867] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:34 2020][8592638.616659] .tg->runnable_avg : 0 [Tue Mar 17 18:02:34 2020][8592638.621453] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:34 2020][8592638.626245] .throttled : 0 [Tue Mar 17 18:02:34 2020][8592638.631037] .throttle_count : 0 [Tue Mar 17 18:02:34 2020][8592638.635830] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:34 2020][8592638.640622] .avg->runnable_avg_period : 47349 [Tue Mar 17 18:02:34 2020][8592638.645766] [Tue Mar 17 18:02:34 2020][8592638.647435] rt_rq[25]:/ [Tue Mar 17 18:02:34 2020][8592638.650061] .rt_nr_running : 0 [Tue Mar 17 18:02:34 2020][8592638.654854] .rt_throttled : 0 [Tue Mar 17 18:02:34 2020][8592638.659647] .rt_time : 0.000000 [Tue Mar 17 18:02:34 2020][8592638.665047] .rt_runtime : 950.000000 [Tue Mar 17 18:02:34 2020][8592638.670619] [Tue Mar 17 18:02:34 2020][8592638.672293] runnable tasks: [Tue Mar 17 18:02:34 2020][8592638.675263] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:34 2020][8592638.686209] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:34 2020][8592638.697094] watchdog/25 136 -11.984090 2148157 0 0.000000 15843.978563 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.708431] migration/25 137 0.000000 30475 0 0.000000 18103.164162 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.719767] ksoftirqd/25 138 81484909.066916 71264028 120 0.000000 851045.899352 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.731104] kworker/25:0H 140 6859.124465 10 100 0.000000 0.041747 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.742494] kdmwork-253:2 12475 81484920.481324 872497173 120 0.000000 30717760.557011 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.753861] kworker/25:1H 18047 81484913.276964 128811140 100 0.000000 671179.811678 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.765256] ptlrpcd_01_06 64823 81484930.312259 4400605 120 0.000000 139188.080782 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.776598] ldlm_cb01_000 66876 222244.090617 25 120 0.000000 0.807214 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.787934] ldlm_cn01_000 66886 81483880.579976 15274226 120 0.000000 321730.561807 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.799288] ll_ost_io01_005 68359 81484281.048625 54631507 120 0.000000 4656629.766047 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.810626] ll_ost_io01_011 68392 81484306.052775 111740152 120 0.000000 7273249.421716 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.821959] ldlm_cn01_003 68471 81483880.583303 18054557 120 0.000000 376488.901387 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.833295] ll_ost01_014 68476 81484913.845290 24671810 120 0.000000 707329.320064 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.844632] ll_ost_io01_013 68489 81484751.532030 76103335 120 0.000000 5571702.970246 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.855975] ll_ost01_024 68570 81484913.256895 18994951 120 0.000000 500055.192445 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.867316] ll_ost01_033 68605 79123115.210607 25561994 120 0.000000 641866.340781 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.878648] ll_ost01_037 68615 81484307.409232 23190392 120 0.000000 600500.616869 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.889988] ll_ost01_050 68671 81484423.365123 23456899 120 0.000000 653055.081663 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.901336] ll_ost_io01_031 68839 81484904.927880 62526317 120 0.000000 5055109.824105 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.912679] ldlm_cn01_004 68910 81483880.583553 15554319 120 0.000000 344859.878211 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.924021] ldlm_cn01_006 69531 81483880.586188 12804734 120 0.000000 286056.823172 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.935353] ldlm_cn01_007 69656 81483864.871013 15793778 120 0.000000 335145.852124 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.946692] ldlm_cb01_003 71785 222244.050542 35 120 0.000000 0.741799 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.958024] ll_ost_io01_057 71823 81484320.358532 16345535 120 0.000000 1225434.202713 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.969368] ll_ost_io01_080 73711 81484748.430161 77426624 120 0.000000 5993411.992512 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.980710] ll_ost_io01_092 73725 81484288.574674 47572717 120 0.000000 3742885.255063 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592638.992046] ll_ost01_088 97767 81002527.800457 27381512 120 0.000000 704420.759261 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.003391] ldlm_bl_56 100137 81484930.268657 6111118 120 0.000000 83763.173055 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.014818] ldlm_bl_93 100174 81484930.270691 6106928 120 0.000000 83514.333712 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.026241] ldlm_bl_96 100177 81484930.268397 6108855 120 0.000000 83946.664272 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.037666] ldlm_cn01_010 105805 81483891.404169 15189545 120 0.000000 318533.960064 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.049086] ldlm_cn01_012 111299 81484924.956604 15279291 120 0.000000 329129.534068 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.060509] ldlm_cn01_015 111302 81483864.874329 17039209 120 0.000000 368849.365569 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.071933] ldlm_cn01_018 111308 81483891.405151 8956046 120 0.000000 200096.710523 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.083356] ldlm_cn01_020 111312 81483891.564320 14949052 120 0.000000 318122.901357 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.094777] ldlm_cn01_022 111314 81483864.874931 14015658 120 0.000000 289771.872506 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.106199] ldlm_cn01_023 111315 81483880.581279 16419287 120 0.000000 346790.001899 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.117627] ldlm_cn01_029 111459 81483880.581329 11154591 120 0.000000 234568.045003 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.129092] ll_ost01_092 101302 66158856.422419 5128963 120 0.000000 173597.825506 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.140513] ll_ost01_096 101306 81484307.411787 20372685 120 0.000000 543077.441493 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.151935] ll_ost01_098 101308 81002527.800247 26200618 120 0.000000 692564.514792 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.163361] ll_ost01_106 101317 79123115.241445 15540426 120 0.000000 427584.891045 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.174805] kworker/25:0 109877 81484927.729502 1155405 120 0.000000 21961.306496 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.186238] kworker/25:2 115897 80969420.471409 2 120 0.000000 0.008355 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.197674] kworker/u258:1 13961 81484930.395867 818 120 0.000000 45.206436 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.209004] kworker/u258:5 14038 81484913.783282 56 120 0.000000 6.927158 0.000000 1 / [Tue Mar 17 18:02:34 2020][8592639.220380] [Tue Mar 17 18:02:34 2020][8592639.222052] cpu#26, 1996.233 MHz [Tue Mar 17 18:02:34 2020][8592639.225458] .nr_running : 0 [Tue Mar 17 18:02:34 2020][8592639.230251] .load : 0 [Tue Mar 17 18:02:34 2020][8592639.235043] .nr_switches : 14076227254 [Tue Mar 17 18:02:34 2020][8592639.240705] .nr_load_updates : 3083218808 [Tue Mar 17 18:02:34 2020][8592639.246299] .nr_uninterruptible : -14552335 [Tue Mar 17 18:02:34 2020][8592639.251789] .next_balance : 12887.283755 [Tue Mar 17 18:02:34 2020][8592639.257558] .curr->pid : 0 [Tue Mar 17 18:02:34 2020][8592639.262353] .clock : 8592639262.278810 [Tue Mar 17 18:02:34 2020][8592639.268532] .cpu_load[0] : 0 [Tue Mar 17 18:02:34 2020][8592639.273339] .cpu_load[1] : 0 [Tue Mar 17 18:02:34 2020][8592639.278135] .cpu_load[2] : 0 [Tue Mar 17 18:02:34 2020][8592639.282941] .cpu_load[3] : 0 [Tue Mar 17 18:02:34 2020][8592639.287753] .cpu_load[4] : 0 [Tue Mar 17 18:02:34 2020][8592639.292549] .avg_idle : 4517766 [Tue Mar 17 18:02:34 2020][8592639.297882] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:34 2020][8592639.303199] [Tue Mar 17 18:02:34 2020][8592639.304867] cfs_rq[26]:/ [Tue Mar 17 18:02:34 2020][8592639.307581] .exec_clock : 0.000000 [Tue Mar 17 18:02:34 2020][8592639.312980] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:34 2020][8592639.318379] .min_vruntime : 131114479.420566 [Tue Mar 17 18:02:34 2020][8592639.324472] .max_vruntime : 0.000001 [Tue Mar 17 18:02:34 2020][8592639.329871] .spread : 0.000000 [Tue Mar 17 18:02:34 2020][8592639.335270] .spread0 : -186100663.912185 [Tue Mar 17 18:02:34 2020][8592639.341450] .nr_spread_over : 0 [Tue Mar 17 18:02:34 2020][8592639.346242] .nr_running : 0 [Tue Mar 17 18:02:34 2020][8592639.351035] .load : 0 [Tue Mar 17 18:02:34 2020][8592639.355827] .runnable_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592639.360621] .blocked_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592639.365412] .tg_load_avg : 0 [Tue Mar 17 18:02:34 2020][8592639.370207] .tg_load_contrib : 0 [Tue Mar 17 18:02:34 2020][8592639.374998] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:34 2020][8592639.379790] .tg->runnable_avg : 0 [Tue Mar 17 18:02:34 2020][8592639.384585] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:34 2020][8592639.389377] .throttled : 0 [Tue Mar 17 18:02:34 2020][8592639.394168] .throttle_count : 0 [Tue Mar 17 18:02:34 2020][8592639.398962] .avg->runnable_avg_sum : 24 [Tue Mar 17 18:02:34 2020][8592639.403840] .avg->runnable_avg_period : 47975 [Tue Mar 17 18:02:34 2020][8592639.408983] [Tue Mar 17 18:02:34 2020][8592639.410653] rt_rq[26]:/ [Tue Mar 17 18:02:34 2020][8592639.413279] .rt_nr_running : 0 [Tue Mar 17 18:02:35 2020][8592639.418073] .rt_throttled : 0 [Tue Mar 17 18:02:35 2020][8592639.422866] .rt_time : 0.006813 [Tue Mar 17 18:02:35 2020][8592639.428264] .rt_runtime : 950.000000 [Tue Mar 17 18:02:35 2020][8592639.433837] [Tue Mar 17 18:02:35 2020][8592639.435510] runnable tasks: [Tue Mar 17 18:02:35 2020][8592639.438483] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:35 2020][8592639.449430] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:35 2020][8592639.460311] watchdog/26 141 -11.985733 2148157 0 0.000000 20233.245592 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.471642] migration/26 142 0.000000 70560 0 0.000000 177275.379874 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.482977] ksoftirqd/26 143 131114407.168775 497254479 120 0.000000 5073181.479420 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.494313] kworker/26:0H 145 128848387.264284 16 100 0.000000 0.111183 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.505769] gssproxy 50218 24604.602275 1 120 0.000000 0.015710 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.517121] monitor_thread 64798 131114467.898848 9098833 120 0.000000 1258823.695235 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.528458] ptlrpcd_02_00 64829 131113972.032777 2449186 120 0.000000 101483.133745 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.539797] ptlrpcd_02_04 64833 131113971.757489 2452522 120 0.000000 101589.197451 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.551162] ll_ost_io02_007 68370 131113997.547182 92976932 120 0.000000 6098743.461897 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.562504] ll_ost02_016 68478 128788132.811237 34415919 120 0.000000 982857.518914 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.573842] ll_ost_io02_014 68490 131113982.247194 149786873 120 0.000000 10155950.704485 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.585184] ll_ost02_022 68549 131113971.401696 46864205 120 0.000000 1255027.629818 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.596522] ll_ost02_025 68557 131114333.026321 46714011 120 0.000000 1355748.675850 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.607876] ll_ost02_060 68732 131114331.723623 41570848 120 0.000000 1222975.311880 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.619213] ll_ost02_067 68763 128788132.811667 47077259 120 0.000000 1362199.060688 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.630547] ll_ost02_070 68781 131114470.123244 41613738 120 0.000000 1218207.723194 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.641884] ll_ost02_072 68787 128672642.941635 38360564 120 0.000000 1089439.280248 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.653225] ll_ost_io02_043 68852 131113948.256633 81051869 120 0.000000 5195877.089485 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.664566] ll_ost_io02_048 68864 131114237.698624 111470960 120 0.000000 7409907.340820 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.675909] ll_ost_create02 71564 130129168.505760 4406654 120 0.000000 83605.847471 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.687253] ll_ost_io02_049 73723 131114036.809899 87275428 120 0.000000 5501170.494367 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.698591] ldlm_bl_10 78686 131114467.429623 6272514 120 0.000000 86226.944559 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.709929] ll_ost02_080 98671 128788132.810376 54166688 120 0.000000 1520366.613916 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.721261] ll_ost02_088 98679 131114467.428581 32816505 120 [8592639.724420] LustreError: dumping log to /tmp/lustre-log.1584493355.68483 [Tue Mar 17 18:02:35 2020] 0.000000 987570.738022 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.739471] ll_ost_io02_057 98928 131114057.179590 136069029 120 0.000000 8933481.560921 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.750812] ldlm_bl_47 100128 131114467.429753 6110126 120 0.000000 83535.118963 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.762240] ldlm_bl_61 100142 131114467.428632 6116591 120 0.000000 83620.548781 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.773679] ldlm_cn02_019 31072 131113739.417973 26578829 120 0.000000 623159.693361 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.785014] ldlm_cn02_022 64115 131114327.026211 31584855 120 0.000000 738287.552365 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.796352] ldlm_cn02_030 64123 131113729.376549 24477625 120 0.000000 585719.244481 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.807685] ldlm_cn02_031 64124 131113730.711256 23126892 120 0.000000 545155.299210 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.819027] ll_ost_io02_065 87190 131114025.842254 120937371 120 0.000000 7769333.481117 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.830372] ll_ost_io02_069 90838 131114247.135690 105672393 120 0.000000 6881987.797900 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.841705] ll_ost_io02_079 90851 131113997.337851 130963952 120 0.000000 8004064.005595 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.853097] kworker/26:2H 18520 131114407.018446 37296344 100 0.000000 157703.139288 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.864439] kworker/26:0 4122 131114467.433681 149518 120 0.000000 914.834507 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.875777] kworker/26:2 10102 131009616.686823 2 120 0.000000 0.009377 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.887113] kworker/u256:3 13933 131114467.601768 1795 120 0.000000 42.820941 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.898453] smtp 14291 131114419.276487 11 120 0.000000 4.897244 0.000000 2 / [Tue Mar 17 18:02:35 2020][8592639.909861] [Tue Mar 17 18:02:35 2020][8592639.911533] cpu#27, 1996.233 MHz [Tue Mar 17 18:02:35 2020][8592639.914941] .nr_running : 0 [Tue Mar 17 18:02:35 2020][8592639.919731] .load : 0 [Tue Mar 17 18:02:35 2020][8592639.924525] .nr_switches : 10538408335 [Tue Mar 17 18:02:35 2020][8592639.930197] .nr_load_updates : 2660214415 [Tue Mar 17 18:02:35 2020][8592639.935775] .nr_uninterruptible : 220430 [Tue Mar 17 18:02:35 2020][8592639.941000] .next_balance : 12887.284444 [Tue Mar 17 18:02:35 2020][8592639.946761] .curr->pid : 0 [Tue Mar 17 18:02:35 2020][8592639.951557] .clock : 8592639951.404148 [Tue Mar 17 18:02:35 2020][8592639.957735] .cpu_load[0] : 0 [Tue Mar 17 18:02:35 2020][8592639.962542] .cpu_load[1] : 0 [Tue Mar 17 18:02:35 2020][8592639.967338] .cpu_load[2] : 0 [Tue Mar 17 18:02:35 2020][8592639.972132] .cpu_load[3] : 0 [Tue Mar 17 18:02:35 2020][8592639.976925] .cpu_load[4] : 0 [Tue Mar 17 18:02:35 2020][8592639.981732] .avg_idle : 10000000 [Tue Mar 17 18:02:35 2020][8592639.987135] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:35 2020][8592639.992454] [Tue Mar 17 18:02:35 2020][8592639.994125] cfs_rq[27]:/ [Tue Mar 17 18:02:35 2020][8592639.996837] .exec_clock : 0.000000 [Tue Mar 17 18:02:35 2020][8592640.002236] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:35 2020][8592640.007635] .min_vruntime : 88738602.257012 [Tue Mar 17 18:02:35 2020][8592640.013642] .max_vruntime : 0.000001 [Tue Mar 17 18:02:35 2020][8592640.019040] .spread : 0.000000 [Tue Mar 17 18:02:35 2020][8592640.024440] .spread0 : -228476541.075739 [Tue Mar 17 18:02:35 2020][8592640.030619] .nr_spread_over : 0 [Tue Mar 17 18:02:35 2020][8592640.035413] .nr_running : 0 [Tue Mar 17 18:02:35 2020][8592640.040204] .load : 0 [Tue Mar 17 18:02:35 2020][8592640.044999] .runnable_load_avg : 0 [Tue Mar 17 18:02:35 2020][8592640.049790] .blocked_load_avg : 0 [Tue Mar 17 18:02:35 2020][8592640.054584] .tg_load_avg : 0 [Tue Mar 17 18:02:35 2020][8592640.059376] .tg_load_contrib : 0 [Tue Mar 17 18:02:35 2020][8592640.064168] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:35 2020][8592640.068961] .tg->runnable_avg : 0 [Tue Mar 17 18:02:35 2020][8592640.073753] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:35 2020][8592640.078546] .throttled : 0 [Tue Mar 17 18:02:35 2020][8592640.083339] .throttle_count : 0 [Tue Mar 17 18:02:35 2020][8592640.088132] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:35 2020][8592640.092925] .avg->runnable_avg_period : 47955 [Tue Mar 17 18:02:35 2020][8592640.098067] [Tue Mar 17 18:02:35 2020][8592640.099737] rt_rq[27]:/ [Tue Mar 17 18:02:35 2020][8592640.102363] .rt_nr_running : 0 [Tue Mar 17 18:02:35 2020][8592640.107156] .rt_throttled : 0 [Tue Mar 17 18:02:35 2020][8592640.111948] .rt_time : 0.000000 [Tue Mar 17 18:02:35 2020][8592640.117347] .rt_runtime : 950.000000 [Tue Mar 17 18:02:35 2020][8592640.122919] [Tue Mar 17 18:02:35 2020][8592640.124592] runnable tasks: [Tue Mar 17 18:02:35 2020][8592640.127565] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:35 2020][8592640.138512] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:35 2020][8592640.149395] watchdog/27 146 -11.986945 2148157 0 0.000000 15597.571347 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.160732] migration/27 147 0.000000 25332 0 0.000000 27710.633169 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.172069] ksoftirqd/27 148 88738414.092357 99925763 120 0.000000 1124516.216331 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.183406] kworker/27:0H 150 1194.957905 10 100 0.000000 0.040466 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.194760] kdevtmpfs 255 229204.381367 1358 120 0.000000 14.123133 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.206110] kworker/27:1H 5596 88738413.936707 163550369 100 0.000000 747025.135782 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.217519] kdmwork-253:68 18336 229204.701581 73913 120 0.000000 903.635798 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.228897] ptlrpcd_03_05 64846 88737974.688372 3587657 120 0.000000 116391.897451 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.240232] ptlrpcd_03_11 64852 88737974.631696 3562393 120 0.000000 115159.371531 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.251584] ll_ost_seq03_00 66995 25326.637384 2 120 0.000000 0.022863 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.262951] ll_ost_io03_017 68504 88737946.676414 91900329 120 0.000000 5383629.893498 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.274295] ll_ost03_027 68572 88738472.913016 21206869 120 0.000000 645920.472566 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.285629] ll_ost03_028 68576 88738414.059825 23069980 120 0.000000 632542.080813 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.296969] ll_ost03_032 68593 88738590.261260 25136290 120 0.000000 743453.642613 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.308307] ll_ost03_045 68640 84075032.316326 23628601 120 0.000000 697554.573302 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.319726] ldlm_cn03_023 64129 88737830.772858 24232886 120 0.000000 546903.261016 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.331064] ll_ost_io03_055 87188 88737981.004922 43237402 120 0.000000 2297063.501049 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.342406] ll_ost_io03_067 90853 88737987.756898 152024790 120 0.000000 8330192.934226 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.353753] ll_ost_io03_076 40251 88738456.583944 89588676 120 0.000000 5885904.266040 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.365103] ll_ost_io03_107 40860 88737986.551134 80614896 120 0.000000 5757727.355006 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.376454] ll_ost03_089 71195 88738593.086157 10753033 120 0.000000 327783.913050 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.387787] ll_ost03_095 74616 88738470.033466 16930118 120 0.000000 436747.381671 0.000000 3 / [Tue Mar 17 18:02:35 2020][8592640.399129] kworker/27:1 80638 88738590.271759 5332037 120 0.000000 83267.979619 0.000000 3 / [Tue Mar 17 18:02:36 2020][8592640.410472] kworker/27:2 11047 84371339.158135 2 120 0.000000 0.012644 0.000000 3 / [Tue Mar 17 18:02:36 2020][8592640.421855] [Tue Mar 17 18:02:36 2020][8592640.423527] cpu#28, 1996.233 MHz [Tue Mar 17 18:02:36 2020][8592640.426935] .nr_running : 0 [Tue Mar 17 18:02:36 2020][8592640.431730] .load : 0 [Tue Mar 17 18:02:36 2020][8592640.436521] .nr_switches : 12634023540 [Tue Mar 17 18:02:36 2020][8592640.442195] .nr_load_updates : 2588198747 [Tue Mar 17 18:02:36 2020][8592640.447770] .nr_uninterruptible : 344690 [Tue Mar 17 18:02:36 2020][8592640.452998] .next_balance : 12887.284956 [Tue Mar 17 18:02:36 2020][8592640.458764] .curr->pid : 0 [Tue Mar 17 18:02:36 2020][8592640.463562] .clock : 8592640463.407145 [Tue Mar 17 18:02:36 2020][8592640.469742] .cpu_load[0] : 0 [Tue Mar 17 18:02:36 2020][8592640.474547] .cpu_load[1] : 0 [Tue Mar 17 18:02:36 2020][8592640.479344] .cpu_load[2] : 0 [Tue Mar 17 18:02:36 2020][8592640.484135] .cpu_load[3] : 0 [Tue Mar 17 18:02:36 2020][8592640.488930] .cpu_load[4] : 0 [Tue Mar 17 18:02:36 2020][8592640.493735] .avg_idle : 10000000 [Tue Mar 17 18:02:36 2020][8592640.499138] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:36 2020][8592640.504450] [Tue Mar 17 18:02:36 2020][8592640.506120] cfs_rq[28]:/ [Tue Mar 17 18:02:36 2020][8592640.508832] .exec_clock : 0.000000 [Tue Mar 17 18:02:36 2020][8592640.514231] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:36 2020][8592640.519631] .min_vruntime : 159162057.042811 [Tue Mar 17 18:02:36 2020][8592640.525725] .max_vruntime : 0.000001 [Tue Mar 17 18:02:36 2020][8592640.531124] .spread : 0.000000 [Tue Mar 17 18:02:36 2020][8592640.536521] .spread0 : -158053086.289940 [Tue Mar 17 18:02:36 2020][8592640.542701] .nr_spread_over : 0 [Tue Mar 17 18:02:36 2020][8592640.547495] .nr_running : 0 [Tue Mar 17 18:02:36 2020][8592640.552287] .load : 0 [Tue Mar 17 18:02:36 2020][8592640.557079] .runnable_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592640.561873] .blocked_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592640.566666] .tg_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592640.571459] .tg_load_contrib : 0 [Tue Mar 17 18:02:36 2020][8592640.576251] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:36 2020][8592640.581044] .tg->runnable_avg : 0 [Tue Mar 17 18:02:36 2020][8592640.585836] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:36 2020][8592640.590629] .throttled : 0 [Tue Mar 17 18:02:36 2020][8592640.595420] .throttle_count : 0 [Tue Mar 17 18:02:36 2020][8592640.600215] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:36 2020][8592640.605006] .avg->runnable_avg_period : 46691 [Tue Mar 17 18:02:36 2020][8592640.610150] [Tue Mar 17 18:02:36 2020][8592640.611819] rt_rq[28]:/ [Tue Mar 17 18:02:36 2020][8592640.614444] .rt_nr_running : 0 [Tue Mar 17 18:02:36 2020][8592640.619238] .rt_throttled : 0 [Tue Mar 17 18:02:36 2020][8592640.624031] .rt_time : 0.000000 [Tue Mar 17 18:02:36 2020][8592640.629429] .rt_runtime : 950.000000 [Tue Mar 17 18:02:36 2020][8592640.635001] [Tue Mar 17 18:02:36 2020][8592640.636674] runnable tasks: [Tue Mar 17 18:02:36 2020][8592640.639649] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:36 2020][8592640.650593] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:36 2020][8592640.661479] watchdog/28 151 -11.987997 2148157 0 0.000000 15347.666468 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.672815] migration/28 152 0.000000 53145 0 0.000000 24597.115636 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.684151] ksoftirqd/28 153 159161996.763351 124630569 120 0.000000 1465057.753817 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.695487] kworker/28:0H 155 5005.174155 10 100 0.000000 0.034547 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.706844] kswapd0 276 159161745.797629 2650538 120 0.000000 15554535.478398 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.718234] kdmwork-253:36 12727 159162008.309404 822730189 120 0.000000 30148350.482917 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.729569] kworker/28:1H 14237 159162045.026455 196226239 100 0.000000 923767.609049 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.740935] kdmwork-253:99 19375 257199.605319 75479 120 0.000000 918.325066 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.752285] bioset 31153 13821.305456 2 100 0.000000 0.011973 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.763633] ptlrpcd_00_02 64807 159161746.346972 4145388 120 0.000000 143367.779838 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.774991] ll_ost00_000 66904 152310032.187838 59734430 120 0.000000 1288511.883534 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.786324] ll_ost_create00 66924 159162045.041019 302102 120 0.000000 3610.465280 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.797701] ll_ost00_023 68583 155802544.881225 53821925 120 0.000000 1155227.163161 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.809043] ll_ost00_032 68623 159162052.627156 38064956 120 0.000000 848446.682544 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.820380] ll_ost00_039 68654 152310032.206103 52235582 120 0.000000 1133613.742566 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.831730] ll_ost00_073 68814 152310032.254954 47731357 120 0.000000 1054846.811393 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.843066] ll_ost_io00_033 68830 159161834.843832 37163634 120 0.000000 2493017.354136 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.854402] ll_ost_io00_035 68838 159161890.883337 99237140 120 0.000000 6943552.173286 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.865738] ll_ost_io00_038 68854 159161764.187878 97235621 120 0.000000 6044104.179418 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.877077] ll_ost_io00_045 68887 159161869.898124 142612444 120 0.000000 9693582.625547 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.888411] ldlm_cn00_005 68911 159161996.852341 20511273 120 0.000000 476590.760703 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.899749] ll_ost_create00 70252 158990872.832461 320477 120 0.000000 3923.500897 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.911124] ldlm_cn00_030 30594 159162045.029528 11662457 120 0.000000 270920.464700 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.922476] ll_ost_io00_059 87559 159161845.119082 146633838 120 0.000000 9526944.120757 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.933815] ll_ost_io00_064 87564 159161996.361381 92837379 120 0.000000 6849212.551866 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.945162] ll_ost_io00_089 40244 159162003.292757 85228838 120 0.000000 5565739.399693 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.956509] ll_ost_io00_100 40828 159161902.346988 86719886 120 0.000000 6133767.281751 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.967844] ll_ost_io00_111 40856 159161878.066848 118738233 120 0.000000 7579982.639426 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.979198] ll_ost00_108 85277 159161999.515660 27155390 120 0.000000 624397.764736 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592640.990537] kworker/28:1 86933 154790176.204151 7 120 0.000000 0.040416 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592641.001878] kworker/28:0 9564 159162045.040138 329107 120 0.000000 5011.048786 0.000000 0 / [Tue Mar 17 18:02:36 2020][8592641.013262] [Tue Mar 17 18:02:36 2020][8592641.014935] cpu#29, 1996.233 MHz [Tue Mar 17 18:02:36 2020][8592641.018345] .nr_running : 0 [Tue Mar 17 18:02:36 2020][8592641.023138] .load : 0 [Tue Mar 17 18:02:36 2020][8592641.027932] .nr_switches : 7988365989 [Tue Mar 17 18:02:36 2020][8592641.033526] .nr_load_updates : 2208366996 [Tue Mar 17 18:02:36 2020][8592641.039104] .nr_uninterruptible : 210153 [Tue Mar 17 18:02:36 2020][8592641.044336] .next_balance : 12887.285544 [Tue Mar 17 18:02:36 2020][8592641.050096] .curr->pid : 0 [Tue Mar 17 18:02:36 2020][8592641.054893] .clock : 8592641054.409809 [Tue Mar 17 18:02:36 2020][8592641.061071] .cpu_load[0] : 0 [Tue Mar 17 18:02:36 2020][8592641.065879] .cpu_load[1] : 0 [Tue Mar 17 18:02:36 2020][8592641.070673] .cpu_load[2] : 0 [Tue Mar 17 18:02:36 2020][8592641.075468] .cpu_load[3] : 0 [Tue Mar 17 18:02:36 2020][8592641.080259] .cpu_load[4] : 0 [Tue Mar 17 18:02:36 2020][8592641.085068] .avg_idle : 6963258 [Tue Mar 17 18:02:36 2020][8592641.090382] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:36 2020][8592641.095695] [Tue Mar 17 18:02:36 2020][8592641.097364] cfs_rq[29]:/ [Tue Mar 17 18:02:36 2020][8592641.100075] .exec_clock : 0.000000 [Tue Mar 17 18:02:36 2020][8592641.105476] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:36 2020][8592641.110876] .min_vruntime : 80740776.177934 [Tue Mar 17 18:02:36 2020][8592641.116882] .max_vruntime : 0.000001 [Tue Mar 17 18:02:36 2020][8592641.122282] .spread : 0.000000 [Tue Mar 17 18:02:36 2020][8592641.127681] .spread0 : -236474367.154817 [Tue Mar 17 18:02:36 2020][8592641.133860] .nr_spread_over : 0 [Tue Mar 17 18:02:36 2020][8592641.138652] .nr_running : 0 [Tue Mar 17 18:02:36 2020][8592641.143446] .load : 0 [Tue Mar 17 18:02:36 2020][8592641.148239] .runnable_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592641.153031] .blocked_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592641.157823] .tg_load_avg : 0 [Tue Mar 17 18:02:36 2020][8592641.162615] .tg_load_contrib : 0 [Tue Mar 17 18:02:36 2020][8592641.167409] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:36 2020][8592641.172200] .tg->runnable_avg : 0 [Tue Mar 17 18:02:36 2020][8592641.176994] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:36 2020][8592641.181787] .throttled : 0 [Tue Mar 17 18:02:36 2020][8592641.186577] .throttle_count : 0 [Tue Mar 17 18:02:36 2020][8592641.191372] .avg->runnable_avg_sum : 20 [Tue Mar 17 18:02:36 2020][8592641.196252] .avg->runnable_avg_period : 48082 [Tue Mar 17 18:02:36 2020][8592641.201392] [Tue Mar 17 18:02:36 2020][8592641.203062] rt_rq[29]:/ [Tue Mar 17 18:02:36 2020][8592641.205690] .rt_nr_running : 0 [Tue Mar 17 18:02:36 2020][8592641.210480] .rt_throttled : 0 [Tue Mar 17 18:02:36 2020][8592641.215274] .rt_time : 0.000000 [Tue Mar 17 18:02:36 2020][8592641.220674] .rt_runtime : 950.000000 [Tue Mar 17 18:02:36 2020][8592641.226245] [Tue Mar 17 18:02:36 2020][8592641.227919] runnable tasks: [Tue Mar 17 18:02:36 2020][8592641.230891] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:36 2020][8592641.241838] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:36 2020][8592641.252722] watchdog/29 156 -11.986133 2148157 0 0.000000 15216.008223 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.264060] migration/29 157 0.000000 28272 0 0.000000 17111.355299 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.275394] ksoftirqd/29 158 80740568.295889 73393622 120 0.000000 873806.386030 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.286732] kworker/29:0H 160 4781.698567 10 100 0.000000 0.039285 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.298146] kworker/29:1H 14238 80740568.282519 132306864 100 0.000000 682050.368957 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.309539] kiblnd_sd_01_02 64788 80740776.403409 6693083404 120 0.000000 50971695.898369 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.320964] ptlrpcd_01_01 64818 80740759.154269 4533377 120 0.000000 136308.818951 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.332311] bioset 66218 25252.257310 2 100 0.000000 0.006863 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.343660] ll_ost01_003 67543 80211081.325034 21650667 120 0.000000 600565.305029 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.355001] ll_ost_io01_006 68362 80740371.123322 103372323 120 0.000000 6446531.777290 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.366344] ll_ost_create01 68470 80000412.046883 3151892 120 0.000000 52982.129048 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.377687] ll_ost_create01 68519 79921584.790565 1623247 120 0.000000 26585.637538 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.389042] ll_ost01_044 68642 80740238.644772 15447309 120 0.000000 408008.953954 0.000000 1 / [Tue Mar 17 18:02:36 2020][8592641.400402] ll_ost_io01_040 68861 80740192.580460 32236621 120 0.000000 2866498.599087 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.411751] ldlm_bl_06 70021 80740762.572541 6330386 120 0.000000 86886.345616 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.423089] ll_ost_io01_064 73695 80740215.039156 32122413 120 0.000000 2201748.701718 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.434445] ldlm_bl_92 100173 80740760.803810 6118454 120 0.000000 83931.364391 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.445871] ldlm_cn01_016 111303 80739811.637215 16120075 120 0.000000 338179.977536 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.457299] ldlm_cn01_025 111324 80739811.636775 15966507 120 0.000000 346032.014021 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.468716] ldlm_cn01_026 111325 80739817.073246 18692197 120 0.000000 410820.893929 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.480210] kworker/29:2 66555 80740623.712366 966051 120 0.000000 17474.224450 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.491550] kworker/29:0 63549 79770440.926159 2 120 0.000000 0.010700 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.502900] kworker/u258:3 13072 80740714.445328 1405 120 0.000000 138.455560 0.000000 1 / [Tue Mar 17 18:02:37 2020][8592641.514265] [Tue Mar 17 18:02:37 2020][8592641.515945] cpu#30, 1996.233 MHz [Tue Mar 17 18:02:37 2020][8592641.519351] .nr_running : 0 [Tue Mar 17 18:02:37 2020][8592641.524144] .load : 0 [Tue Mar 17 18:02:37 2020][8592641.528937] .nr_switches : 13403945055 [Tue Mar 17 18:02:37 2020][8592641.534595] .nr_load_updates : 3001241109 [Tue Mar 17 18:02:37 2020][8592641.540196] .nr_uninterruptible : -14428660 [Tue Mar 17 18:02:37 2020][8592641.545687] .next_balance : 12887.286047 [Tue Mar 17 18:02:37 2020][8592641.551435] .curr->pid : 0 [Tue Mar 17 18:02:37 2020][8592641.556243] .clock : 8592641555.406787 [Tue Mar 17 18:02:37 2020][8592641.562426] .cpu_load[0] : 0 [Tue Mar 17 18:02:37 2020][8592641.567225] .cpu_load[1] : 0 [Tue Mar 17 18:02:37 2020][8592641.572018] .cpu_load[2] : 0 [Tue Mar 17 18:02:37 2020][8592641.576826] .cpu_load[3] : 0 [Tue Mar 17 18:02:37 2020][8592641.581622] .cpu_load[4] : 0 [Tue Mar 17 18:02:37 2020][8592641.586415] .avg_idle : 12075 [Tue Mar 17 18:02:37 2020][8592641.591568] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:37 2020][8592641.596883] [Tue Mar 17 18:02:37 2020][8592641.598551] cfs_rq[30]:/ [Tue Mar 17 18:02:37 2020][8592641.601264] .exec_clock : 0.000000 [Tue Mar 17 18:02:37 2020][8592641.606665] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:37 2020][8592641.612063] .min_vruntime : 133604580.823011 [Tue Mar 17 18:02:37 2020][8592641.618155] .max_vruntime : 0.000001 [Tue Mar 17 18:02:37 2020][8592641.623556] .spread : 0.000000 [Tue Mar 17 18:02:37 2020][8592641.628955] .spread0 : -183610562.509740 [Tue Mar 17 18:02:37 2020][8592641.635135] .nr_spread_over : 0 [Tue Mar 17 18:02:37 2020][8592641.639926] .nr_running : 0 [Tue Mar 17 18:02:37 2020][8592641.644721] .load : 0 [Tue Mar 17 18:02:37 2020][8592641.649512] .runnable_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592641.654304] .blocked_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592641.659098] .tg_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592641.663891] .tg_load_contrib : 0 [Tue Mar 17 18:02:37 2020][8592641.668684] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:37 2020][8592641.673475] .tg->runnable_avg : 0 [Tue Mar 17 18:02:37 2020][8592641.678268] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:37 2020][8592641.683061] .throttled : 0 [Tue Mar 17 18:02:37 2020][8592641.687853] .throttle_count : 0 [Tue Mar 17 18:02:37 2020][8592641.692647] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:37 2020][8592641.697440] .avg->runnable_avg_period : 46118 [Tue Mar 17 18:02:37 2020][8592641.702582] [Tue Mar 17 18:02:37 2020][8592641.704251] rt_rq[30]:/ [Tue Mar 17 18:02:37 2020][8592641.706878] .rt_nr_running : 0 [Tue Mar 17 18:02:37 2020][8592641.711669] .rt_throttled : 0 [Tue Mar 17 18:02:37 2020][8592641.716463] .rt_time : 0.000000 [Tue Mar 17 18:02:37 2020][8592641.721862] .rt_runtime : 950.000000 [Tue Mar 17 18:02:37 2020][8592641.727434] [Tue Mar 17 18:02:37 2020][8592641.729107] runnable tasks: [Tue Mar 17 18:02:37 2020][8592641.732079] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:37 2020][8592641.743027] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:37 2020][8592641.753910] watchdog/30 161 -11.986504 2148157 0 0.000000 20565.303261 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.765248] migration/30 162 0.000000 70508 0 0.000000 182517.896330 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.776583] ksoftirqd/30 163 133604560.670659 518884571 120 0.000000 5224076.098849 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.787919] kworker/30:0H 165 122699757.128457 13 100 0.000000 0.061937 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.799348] kdmwork-253:76 18442 525689.334918 76587 120 0.000000 924.161512 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.810708] writer#4 50898 133604580.823011 49045446 120 0.000000 722495.961186 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.822059] ptlrpcd_02_06 64835 133604428.561792 2448332 120 0.000000 101000.543532 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.833403] jbd2/md0-8 66788 133604434.267735 14591463 120 0.000000 2784203.169569 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.844746] ll_ost_io02_001 66967 133604429.029196 96864709 120 0.000000 6462717.669340 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.856092] ll_ost02_006 68149 132670482.600115 36772277 120 0.000000 1053610.658914 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.867434] ll_ost02_011 68345 133604558.108928 47535687 120 0.000000 1330734.347658 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.878782] ll_ost_io02_017 68501 133604552.716360 167822010 120 0.000000 11400626.805092 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.890134] ll_ost02_036 68604 133604561.750124 28890806 120 0.000000 815743.125068 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.901475] ll_ost02_044 68644 133604558.071398 32825220 120 0.000000 914868.402441 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.912811] ll_ost02_051 68670 133604558.876902 47410791 120 0.000000 1347331.994581 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.924147] ll_ost02_054 68691 133604560.091013 27874082 120 0.000000 784138.701107 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.935481] ll_ost_io02_021 68703 133604561.481468 110508918 120 0.000000 7107467.490336 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.946824] ll_ost_io02_028 68774 133604459.471766 135544039 120 0.000000 9863706.692672 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.958167] ll_ost_io02_035 68832 133604536.996120 163483288 120 0.000000 10878401.519746 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.969520] ldlm_cn02_008 72313 133604249.308537 20203525 120 0.000000 484090.517880 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.980867] ll_ost_io02_051 73726 133604569.134055 82362521 120 0.000000 5311934.813801 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592641.992207] ll_ost_io02_054 97764 133604537.485763 58871607 120 0.000000 3772073.036132 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.003543] ll_ost02_082 98673 132670482.601698 42369877 120 0.000000 1301706.615867 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.014875] ll_ost02_086 98677 131223341.963658 39983362 120 0.000000 1136122.777237 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.026212] ll_ost02_092 98683 133604560.476130 37452125 120 0.000000 1110194.672841 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.037551] ldlm_bl_15 100096 133604568.049980 6116036 120 0.000000 84400.493629 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.048987] ldlm_cb02_004 103708 133095194.881550 2359 120 0.000000 43.905033 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.060421] ldlm_cn02_021 31078 133604249.704624 27327924 120 0.000000 651280.797904 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.071756] ldlm_cn02_025 64118 133546772.174909 21425240 120 0.000000 509192.546856 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.083096] ll_ost_io02_063 87186 133604499.970942 115957831 120 0.000000 7498472.308972 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.094434] ll_ost_io02_074 90843 133604467.517678 164883048 120 0.000000 10542513.405957 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.105773] ll_ost_io02_077 90849 133604440.094502 144313901 120 0.000000 9909083.603833 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.117115] ll_ost_io02_087 90874 133604548.109247 85021643 120 0.000000 5568264.905210 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.128453] ll_ost_io02_088 90875 133604563.430920 126925913 120 0.000000 8392161.522922 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.139788] ll_ost_io02_089 90876 133604485.809494 103293470 120 0.000000 6268800.509630 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.151133] ll_ost_io02_091 40777 133604465.485310 138934974 120 0.000000 8544739.786726 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.162471] ll_ost_io02_093 40779 133604499.153939 57497097 120 0.000000 4223853.115178 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.173805] ll_ost_io02_095 40781 133604512.954096 105809063 120 0.000000 6970008.008478 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.185185] kworker/30:2H 45745 133604557.985401 132065423 100 0.000000 686738.431704 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.196528] kworker/30:0 105246 133604565.452099 178527 120 0.000000 3308.670058 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.207960] kworker/30:1 9194 133461694.611152 4 120 0.000000 0.062308 0.000000 2 / [Tue Mar 17 18:02:37 2020][8592642.219333] [Tue Mar 17 18:02:37 2020][8592642.221007] cpu#31, 1996.233 MHz [Tue Mar 17 18:02:37 2020][8592642.224417] .nr_running : 0 [Tue Mar 17 18:02:37 2020][8592642.229209] .load : 0 [Tue Mar 17 18:02:37 2020][8592642.234001] .nr_switches : 10075669289 [Tue Mar 17 18:02:37 2020][8592642.239660] .nr_load_updates : 2620574052 [Tue Mar 17 18:02:37 2020][8592642.245247] .nr_uninterruptible : 224996 [Tue Mar 17 18:02:37 2020][8592642.250477] .next_balance : 12887.286753 [Tue Mar 17 18:02:37 2020][8592642.256223] .curr->pid : 0 [Tue Mar 17 18:02:37 2020][8592642.261029] .clock : 8592642260.410020 [Tue Mar 17 18:02:37 2020][8592642.267211] .cpu_load[0] : 0 [Tue Mar 17 18:02:37 2020][8592642.272003] .cpu_load[1] : 0 [Tue Mar 17 18:02:37 2020][8592642.276809] .cpu_load[2] : 0 [Tue Mar 17 18:02:37 2020][8592642.281607] .cpu_load[3] : 0 [Tue Mar 17 18:02:37 2020][8592642.286399] .cpu_load[4] : 0 [Tue Mar 17 18:02:37 2020][8592642.291192] .avg_idle : 10000000 [Tue Mar 17 18:02:37 2020][8592642.296607] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:37 2020][8592642.301921] [Tue Mar 17 18:02:37 2020][8592642.303590] cfs_rq[31]:/ [Tue Mar 17 18:02:37 2020][8592642.306304] .exec_clock : 0.000000 [Tue Mar 17 18:02:37 2020][8592642.311703] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:37 2020][8592642.317100] .min_vruntime : 86109538.899628 [Tue Mar 17 18:02:37 2020][8592642.323108] .max_vruntime : 0.000001 [Tue Mar 17 18:02:37 2020][8592642.328508] .spread : 0.000000 [Tue Mar 17 18:02:37 2020][8592642.333906] .spread0 : -231105604.433123 [Tue Mar 17 18:02:37 2020][8592642.340086] .nr_spread_over : 0 [Tue Mar 17 18:02:37 2020][8592642.344880] .nr_running : 0 [Tue Mar 17 18:02:37 2020][8592642.349672] .load : 0 [Tue Mar 17 18:02:37 2020][8592642.354465] .runnable_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592642.359258] .blocked_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592642.364049] .tg_load_avg : 0 [Tue Mar 17 18:02:37 2020][8592642.368841] .tg_load_contrib : 0 [Tue Mar 17 18:02:37 2020][8592642.373635] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:37 2020][8592642.378428] .tg->runnable_avg : 0 [Tue Mar 17 18:02:37 2020][8592642.383221] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:37 2020][8592642.388012] .throttled : 0 [Tue Mar 17 18:02:37 2020][8592642.392804] .throttle_count : 0 [Tue Mar 17 18:02:37 2020][8592642.397600] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:37 2020][8592642.402390] .avg->runnable_avg_period : 46873 [Tue Mar 17 18:02:37 2020][8592642.407533] [Tue Mar 17 18:02:37 2020][8592642.409202] rt_rq[31]:/ [Tue Mar 17 18:02:37 2020][8592642.411830] .rt_nr_running : 0 [Tue Mar 17 18:02:38 2020][8592642.416622] .rt_throttled : 0 [Tue Mar 17 18:02:38 2020][8592642.421415] .rt_time : 0.000000 [Tue Mar 17 18:02:38 2020][8592642.426813] .rt_runtime : 950.000000 [Tue Mar 17 18:02:38 2020][8592642.432386] [Tue Mar 17 18:02:38 2020][8592642.434058] runnable tasks: [Tue Mar 17 18:02:38 2020][8592642.437032] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:38 2020][8592642.447978] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:38 2020][8592642.458866] watchdog/31 166 -11.986334 2148157 0 0.000000 14845.705686 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.470199] migration/31 167 0.000000 23698 0 0.000000 22743.078785 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.481535] ksoftirqd/31 168 86109526.195773 101420184 120 0.000000 1133490.576392 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.492871] kworker/31:0H 170 1403.246711 10 100 0.000000 0.032611 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.504229] kworker/31:1 4738 86109526.948460 9101411 120 0.000000 131740.364818 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.515566] kworker/31:1H 5592 86109526.140375 165608672 100 0.000000 743335.402940 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.526910] multipathd 11206 0.000000 1029 0 0.000000 135.177857 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.538279] kdmwork-253:30 12709 86109526.158461 868101275 120 0.000000 31879803.631434 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.549641] kdmwork-253:75 18430 86109538.140308 813924717 120 0.000000 30242706.917388 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.560989] bioset 21028 6008.143277 2 100 0.000000 0.009357 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.572332] bioset 25233 7202.742509 2 100 0.000000 0.011391 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.583691] ptlrpcd_03_02 64843 86108708.740440 3463409 120 0.000000 120834.552040 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.595039] ldlm_cn03_002 66894 86108410.991967 12292088 120 0.000000 276758.699283 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.606402] ll_ost_io03_014 68497 86109526.186285 80988771 120 0.000000 5187972.940938 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.617741] ll_ost03_020 68545 84946179.059105 20818211 120 0.000000 608934.490983 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.629090] ll_ost03_035 68606 85708876.524365 13075685 120 0.000000 417857.425913 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.640424] ll_ost03_043 68632 81604736.121285 25174038 120 0.000000 711157.228422 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.651774] ll_ost03_065 68765 81604736.121936 22735366 120 0.000000 647963.703183 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.663116] ll_ost_io03_028 68789 86109522.573800 178419661 120 0.000000 10232151.829958 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.674477] ldlm_cn03_009 75136 86107841.754587 33484315 120 0.000000 765707.065084 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.685821] ldlm_bl_26 100107 86109527.269046 6106976 120 0.000000 83588.065097 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.697264] ldlm_cn03_011 122858 86108364.719596 14403970 120 0.000000 308162.567949 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.708681] ldlm_cn03_012 128040 86108364.723102 19724970 120 0.000000 453869.521117 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.720110] ldlm_cn03_014 31070 86108364.721780 16924471 120 0.000000 396930.157767 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.731442] ldlm_cn03_015 31073 86107841.738928 15578370 120 0.000000 337818.627208 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.742783] ldlm_cn03_027 64133 86108364.721459 20513567 120 0.000000 475631.687106 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.754115] ll_ost_io03_051 84744 86108951.944341 85639185 120 0.000000 5272077.553988 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.765461] ll_ost_io03_071 90866 86108946.459733 151978718 120 0.000000 9185374.414843 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.776804] ll_ost_io03_081 40776 86109531.664723 128467803 120 0.000000 7429345.713757 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.788152] ll_ost_io03_095 40837 86108899.278231 107605335 120 0.000000 6348531.132137 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.799486] ll_ost_io03_102 40853 86109528.875799 63734967 120 0.000000 3740670.787325 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.810820] ll_ost_io03_106 40859 86108904.988723 156711317 120 0.000000 9798884.281376 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.822169] ll_ost03_072 70318 81604736.122337 9593523 120 0.000000 288375.121470 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.833510] ll_ost03_101 90276 84946179.064555 5266876 120 0.000000 171784.496807 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.844854] kworker/31:0 96967 84324294.819362 2 120 0.000000 0.007924 0.000000 3 / [Tue Mar 17 18:02:38 2020][8592642.856262] [Tue Mar 17 18:02:38 2020][8592642.857935] cpu#32, 1996.233 MHz [Tue Mar 17 18:02:38 2020][8592642.861341] .nr_running : 0 [Tue Mar 17 18:02:38 2020][8592642.866134] .load : 0 [Tue Mar 17 18:02:38 2020][8592642.870927] .nr_switches : 15611776594 [Tue Mar 17 18:02:38 2020][8592642.876587] .nr_load_updates : 2890399280 [Tue Mar 17 18:02:38 2020][8592642.882175] .nr_uninterruptible : 472006 [Tue Mar 17 18:02:38 2020][8592642.887402] .next_balance : 12887.287386 [Tue Mar 17 18:02:38 2020][8592642.893148] .curr->pid : 0 [Tue Mar 17 18:02:38 2020][8592642.897956] .clock : 8592642897.410671 [Tue Mar 17 18:02:38 2020][8592642.904138] .cpu_load[0] : 0 [Tue Mar 17 18:02:38 2020][8592642.908931] .cpu_load[1] : 0 [Tue Mar 17 18:02:38 2020][8592642.913738] .cpu_load[2] : 0 [Tue Mar 17 18:02:38 2020][8592642.918534] .cpu_load[3] : 0 [Tue Mar 17 18:02:38 2020][8592642.923327] .cpu_load[4] : 0 [Tue Mar 17 18:02:38 2020][8592642.928117] .avg_idle : 5321431 [Tue Mar 17 18:02:38 2020][8592642.933445] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:38 2020][8592642.938761] [Tue Mar 17 18:02:38 2020][8592642.940431] cfs_rq[32]:/ [Tue Mar 17 18:02:38 2020][8592642.943144] .exec_clock : 0.000000 [Tue Mar 17 18:02:38 2020][8592642.948542] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:38 2020][8592642.953941] .min_vruntime : 166205049.352802 [Tue Mar 17 18:02:38 2020][8592642.960033] .max_vruntime : 0.000001 [Tue Mar 17 18:02:38 2020][8592642.965433] .spread : 0.000000 [Tue Mar 17 18:02:38 2020][8592642.970833] .spread0 : -151010093.979949 [Tue Mar 17 18:02:38 2020][8592642.977011] .nr_spread_over : 0 [Tue Mar 17 18:02:38 2020][8592642.981805] .nr_running : 0 [Tue Mar 17 18:02:38 2020][8592642.986598] .load : 0 [Tue Mar 17 18:02:38 2020][8592642.991390] .runnable_load_avg : 0 [Tue Mar 17 18:02:38 2020][8592642.996183] .blocked_load_avg : 0 [Tue Mar 17 18:02:38 2020][8592643.000977] .tg_load_avg : 0 [Tue Mar 17 18:02:38 2020][8592643.005770] .tg_load_contrib : 0 [Tue Mar 17 18:02:38 2020][8592643.010562] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:38 2020][8592643.015355] .tg->runnable_avg : 0 [Tue Mar 17 18:02:38 2020][8592643.020147] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:38 2020][8592643.024940] .throttled : 0 [Tue Mar 17 18:02:38 2020][8592643.029732] .throttle_count : 0 [Tue Mar 17 18:02:38 2020][8592643.034526] .avg->runnable_avg_sum : 72 [Tue Mar 17 18:02:38 2020][8592643.039404] .avg->runnable_avg_period : 46808 [Tue Mar 17 18:02:38 2020][8592643.044546] [Tue Mar 17 18:02:38 2020][8592643.046215] rt_rq[32]:/ [Tue Mar 17 18:02:38 2020][8592643.048843] .rt_nr_running : 0 [Tue Mar 17 18:02:38 2020][8592643.053635] .rt_throttled : 0 [Tue Mar 17 18:02:38 2020][8592643.058428] .rt_time : 0.000000 [Tue Mar 17 18:02:38 2020][8592643.063827] .rt_runtime : 950.000000 [Tue Mar 17 18:02:38 2020][8592643.069401] [Tue Mar 17 18:02:38 2020][8592643.071073] runnable tasks: [Tue Mar 17 18:02:38 2020][8592643.074046] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:38 2020][8592643.084991] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:38 2020][8592643.095879] watchdog/32 171 -11.989330 2148157 0 0.000000 16295.302550 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.107212] migration/32 172 0.000000 586172 0 0.000000 41394.343632 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.118548] ksoftirqd/32 173 166204851.901688 146622017 120 0.000000 1734559.282192 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.129885] kworker/32:0H 175 12013.635370 10 100 0.000000 0.042782 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.141247] ccp-1-q3 6419 2240.920745 2 120 0.000000 0.010499 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.152624] kdmflush 12692 7904.754888 2 100 0.000000 0.009959 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.163974] kworker/32:1H 14221 166204851.888226 228886423 100 0.000000 1021815.322568 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.175312] kdmwork-253:44 18009 567635.606382 67296 120 0.000000 757.142805 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.186694] kiblnd_connd 64781 166205036.667958 12984491 120 0.000000 562600.215915 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.198035] ptlrpcd_00_01 64806 166204655.454709 4204817 120 0.000000 137962.549956 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.209369] ptlrpcd_00_05 64810 166204655.446233 4226565 120 0.000000 138671.043149 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.220713] bioset 65719 66275.072071 2 100 0.000000 0.010158 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.232052] ldiskfs-rsv-con 66697 67541.309667 2 100 0.000000 0.012373 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.243387] ldlm_cn00_001 66884 166204404.574519 16633031 120 0.000000 359513.278615 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.254728] ll_ost00_003 67108 164620665.681809 60017310 120 0.000000 1297219.709115 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.266070] ll_ost00_006 68147 162096159.450533 43970704 120 0.000000 954942.487779 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.277410] ll_ost_io00_003 68353 166204852.019800 123529497 120 0.000000 8388370.715145 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.288771] ll_ost00_022 68578 162096158.635975 35231115 120 0.000000 786510.386504 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.300110] ll_ost00_026 68591 166205049.561648 44021491 120 0.000000 1003109.776852 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.311444] ll_ost00_028 68600 162096158.560744 31398051 120 0.000000 679580.521151 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.322792] ll_ost00_054 68724 166204990.797185 60152330 120 0.000000 1304530.926810 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.334128] ll_ost_io00_026 68750 166204820.517506 97847797 120 0.000000 6399161.420699 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.345470] ll_ost_io00_036 68842 166204799.116318 78894317 120 0.000000 5388234.721412 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.356813] ll_ost_io00_047 68894 166204653.756538 140667560 120 0.000000 10186870.136601 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.368150] ll_ost_io00_049 68897 166204749.824958 89267544 120 0.000000 6448414.264596 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.379529] ldlm_cn00_013 111318 166204404.573487 14458961 120 0.000000 335764.347511 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.390956] ldlm_cn00_020 30584 166204404.560654 26347547 120 0.000000 624449.056627 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.402291] ldlm_cn00_028 30592 166204969.397525 20920350 120 0.000000 470075.506721 0.000000 0 / [Tue Mar 17 18:02:38 2020][8592643.413641] ll_ost_io00_061 87561 166204851.945992 116064445 120 0.000000 8054047.368042 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.425004] ll_ost_io00_103 40840 166204717.802589 103201153 120 0.000000 6101091.017788 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.436345] ll_ost_io00_109 40852 166204711.505572 88614398 120 0.000000 5827013.987913 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.447690] ll_ost00_087 101337 166205014.344990 49896339 120 0.000000 1086998.618688 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.459116] ll_ost00_094 85259 162096158.557527 16858469 120 0.000000 395251.219259 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.470459] ll_ost00_111 125736 162096158.752124 22386926 120 0.000000 507448.485489 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.481902] kworker/32:0 4626 166205036.628916 32256 120 0.000000 353.620072 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.493238] kworker/u257:2 4710 166201967.872258 36881 120 0.000000 2738.661400 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.504577] kworker/32:2 9705 165641587.398878 2 120 0.000000 0.009367 0.000000 0 / [Tue Mar 17 18:02:39 2020][8592643.515944] [Tue Mar 17 18:02:39 2020][8592643.517619] cpu#33, 1996.233 MHz [Tue Mar 17 18:02:39 2020][8592643.521029] .nr_running : 0 [Tue Mar 17 18:02:39 2020][8592643.525827] .load : 0 [Tue Mar 17 18:02:39 2020][8592643.530621] .nr_switches : 8301300063 [Tue Mar 17 18:02:39 2020][8592643.536210] .nr_load_updates : 2270627095 [Tue Mar 17 18:02:39 2020][8592643.541800] .nr_uninterruptible : 216996 [Tue Mar 17 18:02:39 2020][8592643.547029] .next_balance : 12887.288048 [Tue Mar 17 18:02:39 2020][8592643.552810] .curr->pid : 0 [Tue Mar 17 18:02:39 2020][8592643.557610] .clock : 8592643557.412735 [Tue Mar 17 18:02:39 2020][8592643.563789] .cpu_load[0] : 0 [Tue Mar 17 18:02:39 2020][8592643.568597] .cpu_load[1] : 0 [Tue Mar 17 18:02:39 2020][8592643.573391] .cpu_load[2] : 0 [Tue Mar 17 18:02:39 2020][8592643.578184] .cpu_load[3] : 0 [Tue Mar 17 18:02:39 2020][8592643.582978] .cpu_load[4] : 0 [Tue Mar 17 18:02:39 2020][8592643.587783] .avg_idle : 4039104 [Tue Mar 17 18:02:39 2020][8592643.593100] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:39 2020][8592643.598410] [Tue Mar 17 18:02:39 2020][8592643.600080] cfs_rq[33]:/ [Tue Mar 17 18:02:39 2020][8592643.602795] .exec_clock : 0.000000 [Tue Mar 17 18:02:39 2020][8592643.608194] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:39 2020][8592643.613593] .min_vruntime : 92781506.262253 [Tue Mar 17 18:02:39 2020][8592643.619598] .max_vruntime : 0.000001 [Tue Mar 17 18:02:39 2020][8592643.624998] .spread : 0.000000 [Tue Mar 17 18:02:39 2020][8592643.630397] .spread0 : -224433637.070498 [Tue Mar 17 18:02:39 2020][8592643.636577] .nr_spread_over : 0 [Tue Mar 17 18:02:39 2020][8592643.641370] .nr_running : 0 [Tue Mar 17 18:02:39 2020][8592643.646161] .load : 0 [Tue Mar 17 18:02:39 2020][8592643.650955] .runnable_load_avg : 0 [Tue Mar 17 18:02:39 2020][8592643.655747] .blocked_load_avg : 0 [Tue Mar 17 18:02:39 2020][8592643.660542] .tg_load_avg : 0 [Tue Mar 17 18:02:39 2020][8592643.665334] .tg_load_contrib : 0 [Tue Mar 17 18:02:39 2020][8592643.670127] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:39 2020][8592643.674918] .tg->runnable_avg : 0 [Tue Mar 17 18:02:39 2020][8592643.679710] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:39 2020][8592643.684502] .throttled : 0 [Tue Mar 17 18:02:39 2020][8592643.689296] .throttle_count : 0 [Tue Mar 17 18:02:39 2020][8592643.694090] .avg->runnable_avg_sum : 6 [Tue Mar 17 18:02:39 2020][8592643.698881] .avg->runnable_avg_period : 46248 [Tue Mar 17 18:02:39 2020][8592643.704023] [Tue Mar 17 18:02:39 2020][8592643.705693] rt_rq[33]:/ [Tue Mar 17 18:02:39 2020][8592643.708319] .rt_nr_running : 0 [Tue Mar 17 18:02:39 2020][8592643.713112] .rt_throttled : 0 [Tue Mar 17 18:02:39 2020][8592643.717905] .rt_time : 0.006191 [Tue Mar 17 18:02:39 2020][8592643.723305] .rt_runtime : 950.000000 [Tue Mar 17 18:02:39 2020][8592643.728876] [Tue Mar 17 18:02:39 2020][8592643.730549] runnable tasks: [Tue Mar 17 18:02:39 2020][8592643.733522] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:39 2020][8592643.744467] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:39 2020][8592643.755355] watchdog/33 176 -11.985101 2148160 0 0.000000 15236.939303 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.766690] migration/33 177 0.000000 508499 0 0.000000 19461.690968 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.778025] ksoftirqd/33 178 92780241.247572 71947355 120 0.000000 919594.288555 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.789361] kworker/33:0H 180 12830.786175 10 100 0.000000 0.042520 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.800712] netns 256 47.370909 2 100 0.000000 0.010359 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.812042] kintegrityd 259 83.396851 2 100 0.000000 0.009067 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.823380] bioset 260 95.405508 2 100 0.000000 0.009357 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.834715] bioset 261 107.413714 2 100 0.000000 0.008916 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.846051] bioset 262 119.421529 2 100 0.000000 0.008516 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.857388] kblockd 263 131.429384 2 100 0.000000 0.008536 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.868723] md 264 143.437951 2 100 0.000000 0.009218 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.880060] edac-poller 265 155.446903 2 100 0.000000 0.009880 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.891396] watchdogd 266 167.454778 2 100 0.000000 0.008476 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.902732] irq/27-AMD-Vi 269 0.000000 2 49 0.000000 0.018094 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.914069] ksmd 280 335.729885 2 125 0.000000 0.010750 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.925404] crypto 282 359.834426 2 100 0.000000 0.009567 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.936740] kthrotld 290 456.063868 2 100 0.000000 0.010349 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.948075] kmpath_rdacd 295 516.112207 2 100 0.000000 0.019105 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.959411] kaluad 296 528.127755 2 100 0.000000 0.016001 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.970749] ipv6_addrconf 301 564.172448 2 100 0.000000 0.017614 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.982083] deferwq 315 636.320594 2 100 0.000000 0.011712 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592643.993420] scsi_tmf_0 580 1213.389123 2 100 0.000000 0.011642 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.004758] mlx5_page_alloc 2113 1556.971886 2 100 0.000000 0.009398 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.016103] ccp-3-q2 6476 3157.145746 2 120 0.000000 0.012022 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.027490] kworker/33:1H 18026 92780255.635546 126806768 100 0.000000 650215.906480 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.038831] kdmwork-253:51 18135 92780255.657793 870004789 120 0.000000 31212752.970757 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.050177] kdmwork-253:66 18324 92780267.635441 807074131 120 0.000000 28650258.290045 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.061532] kvm-irqfd-clean 19703 14585.488471 2 100 0.000000 0.011963 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.072899] ptlrpcd_01_03 64820 92781486.825056 4544730 120 0.000000 135788.160736 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.084249] ll_ost01_000 66908 92779953.284273 21102324 120 0.000000 574942.177733 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.095586] ll_ost_io01_000 66952 92779687.310001 69265246 120 0.000000 4719900.043490 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.106919] ll_ost_io01_001 66953 92779960.131394 77619350 120 0.000000 5871114.584965 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.118256] ll_ost_io01_002 66954 92779963.763719 57444672 120 0.000000 3431985.501486 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.129599] ll_ost01_005 67860 92780561.163225 25283235 120 0.000000 653007.246915 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.140953] ll_ost01_013 68472 74419024.649890 13979169 120 0.000000 334119.577785 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.152304] ll_ost01_020 68558 92154947.165896 20436304 120 0.000000 542209.676525 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.163638] ll_ost01_022 68567 92780637.618846 19734232 120 0.000000 493576.137218 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.174981] ll_ost01_042 68635 89470827.371657 16977747 120 0.000000 446327.893463 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.186314] ll_ost01_043 68637 92780163.562391 17886685 120 0.000000 441879.524817 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.197656] ll_ost01_051 68674 92780443.560178 17234989 120 0.000000 411074.088075 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.208990] ll_ost01_054 68681 92781501.803345 18497630 120 0.000000 490905.252839 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.220330] ll_ost01_066 68736 92780991.479903 22035109 120 0.000000 609327.940265 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.231664] ll_ost01_071 68779 74419024.649910 12156851 120 0.000000 299593.564170 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.243005] ll_ost_io01_034 68846 92780162.084943 12540861 120 0.000000 908888.042026 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.254342] ll_ost_io01_036 68851 92779761.790653 132045855 120 0.000000 8156619.988670 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.265677] ll_ost_io01_038 68856 92780250.870081 35802876 120 0.000000 2683423.922272 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.277022] ldlm_cb01_002 71784 292239.600198 32 120 0.000000 0.606285 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.288358] ll_ost_io01_056 71822 92780201.474229 76490409 120 0.000000 5216635.373373 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.299696] ll_ost_io01_067 73698 92779965.170229 64487264 120 0.000000 4940545.774551 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.311039] ldlm_cb01_004 76900 292239.621147 23 120 0.000000 0.395333 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.322375] ll_ost01_083 97757 92780635.268340 25202064 120 0.000000 685357.532685 0.000000[8592644.332432] LustreError: dumping log to /tmp/lustre-log.1584493359.68804 [Tue Mar 17 18:02:39 2020] 1 / [Tue Mar 17 18:02:39 2020][8592644.340658] ll_ost01_100 101310 92781506.487477 13718012 120 0.000000 353913.163516 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.352109] kworker/33:2 4521 92781494.147559 29685 120 0.000000 419.132865 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.363448] kworker/33:1 5248 92698372.424199 3 120 0.000000 0.014308 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.374794] sleep 14300 92781122.701757 2 120 0.000000 0.617944 0.000000 1 / [Tue Mar 17 18:02:39 2020][8592644.386166] [Tue Mar 17 18:02:39 2020][8592644.387838] cpu#34, 1996.233 MHz [Tue Mar 17 18:02:39 2020][8592644.391245] .nr_running : 0 [Tue Mar 17 18:02:39 2020][8592644.396036] .load : 0 [Tue Mar 17 18:02:39 2020][8592644.400829] .nr_switches : 14114225858 [Tue Mar 17 18:02:39 2020][8592644.406489] .nr_load_updates : 3101699882 [Tue Mar 17 18:02:39 2020][8592644.412077] .nr_uninterruptible : -15828430 [Tue Mar 17 18:02:39 2020][8592644.417566] .next_balance : 12887.288918 [Tue Mar 17 18:02:39 2020][8592644.423311] .curr->pid : 0 [Tue Mar 17 18:02:40 2020][8592644.428119] .clock : 8592644427.419442 [Tue Mar 17 18:02:40 2020][8592644.434301] .cpu_load[0] : 0 [Tue Mar 17 18:02:40 2020][8592644.439093] .cpu_load[1] : 0 [Tue Mar 17 18:02:40 2020][8592644.443887] .cpu_load[2] : 0 [Tue Mar 17 18:02:40 2020][8592644.448706] .cpu_load[3] : 0 [Tue Mar 17 18:02:40 2020][8592644.453498] .cpu_load[4] : 0 [Tue Mar 17 18:02:40 2020][8592644.458290] .avg_idle : 7192391 [Tue Mar 17 18:02:40 2020][8592644.463618] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:40 2020][8592644.468932] [Tue Mar 17 18:02:40 2020][8592644.470602] cfs_rq[34]:/ [Tue Mar 17 18:02:40 2020][8592644.473315] .exec_clock : 0.000000 [Tue Mar 17 18:02:40 2020][8592644.478713] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:40 2020][8592644.484114] .min_vruntime : 151491678.357817 [Tue Mar 17 18:02:40 2020][8592644.490207] .max_vruntime : 0.000001 [Tue Mar 17 18:02:40 2020][8592644.495605] .spread : 0.000000 [Tue Mar 17 18:02:40 2020][8592644.501004] .spread0 : -165723464.974934 [Tue Mar 17 18:02:40 2020][8592644.507183] .nr_spread_over : 0 [Tue Mar 17 18:02:40 2020][8592644.511977] .nr_running : 0 [Tue Mar 17 18:02:40 2020][8592644.516768] .load : 0 [Tue Mar 17 18:02:40 2020][8592644.521561] .runnable_load_avg : 0 [Tue Mar 17 18:02:40 2020][8592644.526354] .blocked_load_avg : 51 [Tue Mar 17 18:02:40 2020][8592644.531233] .tg_load_avg : 0 [Tue Mar 17 18:02:40 2020][8592644.536028] .tg_load_contrib : 0 [Tue Mar 17 18:02:40 2020][8592644.540821] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:40 2020][8592644.545614] .tg->runnable_avg : 0 [Tue Mar 17 18:02:40 2020][8592644.550404] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:40 2020][8592644.555197] .throttled : 0 [Tue Mar 17 18:02:40 2020][8592644.559992] .throttle_count : 0 [Tue Mar 17 18:02:40 2020][8592644.564782] .avg->runnable_avg_sum : 44 [Tue Mar 17 18:02:40 2020][8592644.569662] .avg->runnable_avg_period : 47846 [Tue Mar 17 18:02:40 2020][8592644.574806] [Tue Mar 17 18:02:40 2020][8592644.576476] rt_rq[34]:/ [Tue Mar 17 18:02:40 2020][8592644.579101] .rt_nr_running : 0 [Tue Mar 17 18:02:40 2020][8592644.583892] .rt_throttled : 0 [Tue Mar 17 18:02:40 2020][8592644.588686] .rt_time : 0.000000 [Tue Mar 17 18:02:40 2020][8592644.594085] .rt_runtime : 950.000000 [Tue Mar 17 18:02:40 2020][8592644.599659] [Tue Mar 17 18:02:40 2020][8592644.601330] runnable tasks: [Tue Mar 17 18:02:40 2020][8592644.604304] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:40 2020][8592644.615249] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:40 2020][8592644.626138] watchdog/34 181 -11.985953 2148161 0 0.000000 20774.760745 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.637471] migration/34 182 0.000000 547516 0 0.000000 187547.188068 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.648806] ksoftirqd/34 183 151491630.297522 549809331 120 0.000000 5649528.405432 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.660159] kpsmoused 299 35.000225 2 100 0.000000 0.019155 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.671498] scsi_tmf_1 1961 861.734735 2 100 0.000000 0.014006 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.682839] kworker/34:1H 5589 145303018.755722 943540569 100 0.000000 4560234.398954 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.694185] ccp-5-q2 6484 2912.399099 2 120 0.000000 0.008787 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.705520] ccp-5-q3 6485 2924.407067 2 120 0.000000 0.007974 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.716859] ccp-5-q4 6486 2936.414916 2 120 0.000000 0.007855 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.728265] kdmflush 18491 12878.324105 2 100 0.000000 0.010279 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.739625] smartd 50173 151465462.620748 2508807 120 0.000000 30011.989415 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.750975] kiblnd_sd_02_01 64791 151491676.596201 11153105033 120 0.000000 88713057.649039 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.762489] ptlrpcd_02_10 64839 151489521.675005 2451084 120 0.000000 101721.118735 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.773836] ldlm_cn02_001 66890 151489292.174574 25557246 120 0.000000 615762.091842 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.785168] ldlm_cn02_002 66891 151489292.174535 23757228 120 0.000000 539748.776833 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.796515] ll_ost02_004 67611 151491176.594956 33897657 120 0.000000 972316.498248 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.807886] ll_ost02_031 68590 151491461.672819 23507980 120 0.000000 668104.032360 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.819233] ll_ost02_058 68717 151491666.359390 40553502 120 0.000000 1137550.677735 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.830570] ll_ost02_062 68740 151491392.894418 24699645 120 0.000000 706790.698175 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.841907] ll_ost02_070 68781 151491666.415866 41614111 120 0.000000 1218212.833969 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.853266] ll_ost_io02_050 73724 151489511.749475 143525129 120 0.000000 8125007.486492 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.864602] ldlm_cn02_009 90301 151489292.174725 14648434 120 0.000000 341611.133864 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.875938] ll_ost_io02_053 97762 151489739.508744 123796504 120 0.000000 7972679.276570 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.887276] ll_ost02_089 98680 151491666.361454 42269855 120 0.000000 1129282.251794 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.898621] ldlm_bl_67 100148 151491654.346461 6110796 120 0.000000 83523.943204 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.910071] ll_ost_io02_078 90850 151489736.092621 111248807 120 0.000000 7402149.198350 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.921422] ll_ost_io02_104 40807 151489759.246546 154998502 120 0.000000 9866772.810886 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.932756] ll_ost_io02_107 40819 151489559.893166 181094298 120 0.000000 11381024.489975 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.944106] ll_ost02_104 101332 151491666.360702 37271689 120 0.000000 1105521.188570 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.955554] kworker/34:2H 39994 151491630.271453 83430914 100 0.000000 352601.772534 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.966902] kworker/34:2 4655 151491654.344096 83102 120 0.000000 960.004027 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.978246] kworker/34:0 13494 151465462.033475 46 120 0.000000 0.180767 0.000000 2 / [Tue Mar 17 18:02:40 2020][8592644.989625] [Tue Mar 17 18:02:40 2020][8592644.991300] cpu#35, 1996.233 MHz [Tue Mar 17 18:02:40 2020][8592644.994711] .nr_running : 0 [Tue Mar 17 18:02:40 2020][8592644.999510] .load : 0 [Tue Mar 17 18:02:40 2020][8592645.004304] .nr_switches : 10142058051 [Tue Mar 17 18:02:40 2020][8592645.009975] .nr_load_updates : 2604618015 [Tue Mar 17 18:02:40 2020][8592645.015551] .nr_uninterruptible : 245454 [Tue Mar 17 18:02:40 2020][8592645.020777] .next_balance : 12887.289523 [Tue Mar 17 18:02:40 2020][8592645.026539] .curr->pid : 0 [Tue Mar 17 18:02:40 2020][8592645.031335] .clock : 8592645030.417930 [Tue Mar 17 18:02:40 2020][8592645.037516] .cpu_load[0] : 0 [Tue Mar 17 18:02:40 2020][8592645.042328] .cpu_load[1] : 0 [Tue Mar 17 18:02:40 2020][8592645.047125] .cpu_load[2] : 0 [Tue Mar 17 18:02:40 2020][8592645.051931] .cpu_load[3] : 0 [Tue Mar 17 18:02:40 2020][8592645.056742] .cpu_load[4] : 0 [Tue Mar 17 18:02:40 2020][8592645.061538] .avg_idle : 10000000 [Tue Mar 17 18:02:40 2020][8592645.066951] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:40 2020][8592645.072267] [Tue Mar 17 18:02:40 2020][8592645.073937] cfs_rq[35]:/ [Tue Mar 17 18:02:40 2020][8592645.076648] .exec_clock : 0.000000 [Tue Mar 17 18:02:40 2020][8592645.082050] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:40 2020][8592645.087448] .min_vruntime : 100870339.965848 [Tue Mar 17 18:02:40 2020][8592645.093539] .max_vruntime : 0.000001 [Tue Mar 17 18:02:40 2020][8592645.098939] .spread : 0.000000 [Tue Mar 17 18:02:40 2020][8592645.104338] .spread0 : -216344803.366903 [Tue Mar 17 18:02:40 2020][8592645.110519] .nr_spread_over : 0 [Tue Mar 17 18:02:40 2020][8592645.115312] .nr_running : 0 [Tue Mar 17 18:02:40 2020][8592645.120105] .load : 0 [Tue Mar 17 18:02:40 2020][8592645.124896] .runnable_load_avg : 0 [Tue Mar 17 18:02:40 2020][8592645.129688] .blocked_load_avg : 0 [Tue Mar 17 18:02:40 2020][8592645.134482] .tg_load_avg : 0 [Tue Mar 17 18:02:40 2020][8592645.139275] .tg_load_contrib : 0 [Tue Mar 17 18:02:40 2020][8592645.144068] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:40 2020][8592645.148860] .tg->runnable_avg : 0 [Tue Mar 17 18:02:40 2020][8592645.153653] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:40 2020][8592645.158445] .throttled : 0 [Tue Mar 17 18:02:40 2020][8592645.163237] .throttle_count : 0 [Tue Mar 17 18:02:40 2020][8592645.168031] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:40 2020][8592645.172823] .avg->runnable_avg_period : 46959 [Tue Mar 17 18:02:40 2020][8592645.177965] [Tue Mar 17 18:02:40 2020][8592645.179635] rt_rq[35]:/ [Tue Mar 17 18:02:40 2020][8592645.182261] .rt_nr_running : 0 [Tue Mar 17 18:02:40 2020][8592645.187055] .rt_throttled : 0 [Tue Mar 17 18:02:40 2020][8592645.191848] .rt_time : 0.000000 [Tue Mar 17 18:02:40 2020][8592645.197246] .rt_runtime : 950.000000 [Tue Mar 17 18:02:40 2020][8592645.202818] [Tue Mar 17 18:02:40 2020][8592645.204492] runnable tasks: [Tue Mar 17 18:02:40 2020][8592645.207465] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:40 2020][8592645.218410] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:40 2020][8592645.229300] watchdog/35 186 -11.983710 2148160 0 0.000000 14553.133608 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.240632] migration/35 187 0.000000 630964 0 0.000000 26629.495865 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.251967] ksoftirqd/35 188 100868401.077033 101170798 120 0.000000 1179194.508342 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.263304] kworker/35:0H 190 2496.359407 10 100 0.000000 0.035987 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.274660] kworker/35:1H 5588 100868399.866008 161503060 100 0.000000 722187.322047 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.286049] kdmflush 17997 13794.031676 2 100 0.000000 0.008396 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.297415] kdmwork-253:108 24528 438445.636712 84520 120 0.000000 1030.594723 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.308762] gssproxy 50219 37291.499502 1 120 0.000000 0.016681 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.320097] gssproxy 50220 37303.506750 1 120 0.000000 0.007254 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.331432] gssproxy 50221 37315.514178 1 120 0.000000 0.007434 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.342790] raid5wq 65596 66169.118620 2 100 0.000000 0.011231 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.354123] bioset 65598 66181.128527 2 100 0.000000 0.011121 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.365463] ldiskfs-rsv-con 66499 67364.309939 2 100 0.000000 0.010850 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.376803] ll_ost03_001 66922 95328403.687797 12414211 120 0.000000 382896.244365 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.388140] ll_ost03_002 66923 100870329.046584 17196290 120 0.000000 497350.988232 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.399480] ll_ost_out03_00 67050 78070231.772344 57 120 0.000000 0.995757 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.410823] ll_ost03_009 68352 100408023.682135 22598696 120 0.000000 624268.531852 0.000000 3 / [Tue Mar 17 18:02:40 2020][8592645.422175] ll_ost03_014 68534 99231661.706787 22120370 120 0.000000 656709.437287 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.433520] ll_ost03_030 68588 95328403.687977 20502115 120 0.000000 568926.891287 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.444858] ll_ost03_036 68607 100868873.095335 10196569 120 0.000000 323819.025893 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.456192] ll_ost03_042 68627 100868825.418839 29312058 120 0.000000 829148.008577 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.467533] ll_ost03_053 68669 95328403.688488 21631102 120 0.000000 603022.602651 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.478875] ll_ost03_057 68680 95328403.688167 15722858 120 0.000000 476335.613623 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.490233] ll_ost_create03 68907 99881730.079767 2290152 120 0.000000 39182.827384 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.501599] ldlm_bl_50 100131 100870327.993038 6112352 120 0.000000 83558.113532 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.513047] ldlm_cn03_030 64136 100867384.770550 13540278 120 0.000000 293913.741075 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.524424] ll_ost03_068 55698 95328403.687746 11732571 120 0.000000 363371.256942 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.535761] ll_ost03_071 70317 100868861.884541 7567293 120 0.000000 208010.572231 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.547097] ll_ost03_076 70322 100868886.106664 5884255 120 0.000000 187682.832686 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.558434] ll_ost03_088 71194 95328403.688247 6034217 120 0.000000 225833.868216 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.569771] ll_ost03_093 74613 100868862.203442 9231145 120 0.000000 269486.758414 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.581124] md4_resync 123029 100867713.566049 98703266 125 0.000000 3990485.329729 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.592551] kworker/35:2 130919 100778395.738589 27 120 0.000000 0.097343 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.603985] kworker/35:0 11937 100870327.979513 6823 120 0.000000 117.513709 0.000000 3 / [Tue Mar 17 18:02:41 2020][8592645.615367] [Tue Mar 17 18:02:41 2020][8592645.617039] cpu#36, 1996.233 MHz [Tue Mar 17 18:02:41 2020][8592645.620447] .nr_running : 0 [Tue Mar 17 18:02:41 2020][8592645.625238] .load : 0 [Tue Mar 17 18:02:41 2020][8592645.630030] .nr_switches : 8593126029 [Tue Mar 17 18:02:41 2020][8592645.635605] .nr_load_updates : 2174626602 [Tue Mar 17 18:02:41 2020][8592645.641191] .nr_uninterruptible : 471378 [Tue Mar 17 18:02:41 2020][8592645.646423] .next_balance : 12887.290148 [Tue Mar 17 18:02:41 2020][8592645.652177] .curr->pid : 0 [Tue Mar 17 18:02:41 2020][8592645.656996] .clock : 8592645656.418813 [Tue Mar 17 18:02:41 2020][8592645.663181] .cpu_load[0] : 0 [Tue Mar 17 18:02:41 2020][8592645.667976] .cpu_load[1] : 0 [Tue Mar 17 18:02:41 2020][8592645.672780] .cpu_load[2] : 0 [Tue Mar 17 18:02:41 2020][8592645.677577] .cpu_load[3] : 0 [Tue Mar 17 18:02:41 2020][8592645.682370] .cpu_load[4] : 0 [Tue Mar 17 18:02:41 2020][8592645.687164] .avg_idle : 10000000 [Tue Mar 17 18:02:41 2020][8592645.692578] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:41 2020][8592645.697892] [Tue Mar 17 18:02:41 2020][8592645.699561] cfs_rq[36]:/ [Tue Mar 17 18:02:41 2020][8592645.702274] .exec_clock : 0.000000 [Tue Mar 17 18:02:41 2020][8592645.707674] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:41 2020][8592645.713073] .min_vruntime : 177931088.309963 [Tue Mar 17 18:02:41 2020][8592645.719165] .max_vruntime : 0.000001 [Tue Mar 17 18:02:41 2020][8592645.724565] .spread : 0.000000 [Tue Mar 17 18:02:41 2020][8592645.729964] .spread0 : -139284055.022788 [Tue Mar 17 18:02:41 2020][8592645.736144] .nr_spread_over : 0 [Tue Mar 17 18:02:41 2020][8592645.740938] .nr_running : 0 [Tue Mar 17 18:02:41 2020][8592645.745730] .load : 0 [Tue Mar 17 18:02:41 2020][8592645.750523] .runnable_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592645.755313] .blocked_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592645.760108] .tg_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592645.764900] .tg_load_contrib : 0 [Tue Mar 17 18:02:41 2020][8592645.769693] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:41 2020][8592645.774485] .tg->runnable_avg : 0 [Tue Mar 17 18:02:41 2020][8592645.779277] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:41 2020][8592645.784071] .throttled : 0 [Tue Mar 17 18:02:41 2020][8592645.788862] .throttle_count : 0 [Tue Mar 17 18:02:41 2020][8592645.793657] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:41 2020][8592645.798449] .avg->runnable_avg_period : 46621 [Tue Mar 17 18:02:41 2020][8592645.803592] [Tue Mar 17 18:02:41 2020][8592645.805261] rt_rq[36]:/ [Tue Mar 17 18:02:41 2020][8592645.807887] .rt_nr_running : 0 [Tue Mar 17 18:02:41 2020][8592645.812680] .rt_throttled : 0 [Tue Mar 17 18:02:41 2020][8592645.817472] .rt_time : 0.000000 [Tue Mar 17 18:02:41 2020][8592645.822872] .rt_runtime : 950.000000 [Tue Mar 17 18:02:41 2020][8592645.828443] [Tue Mar 17 18:02:41 2020][8592645.830115] runnable tasks: [Tue Mar 17 18:02:41 2020][8592645.833088] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:41 2020][8592645.844036] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:41 2020][8592645.854927] watchdog/36 191 -11.988749 2148159 0 0.000000 16353.136545 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.866257] migration/36 192 0.000000 587903 0 0.000000 60672.152397 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.877594] ksoftirqd/36 193 177930092.960749 163817572 120 0.000000 1839057.674986 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.888928] kworker/36:0H 195 8430.891577 10 100 0.000000 0.048220 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.900325] kdmwork-253:21 12682 177930154.651102 823656319 120 0.000000 29489458.422915 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.911674] kworker/36:1H 14204 177931076.310004 247701079 100 0.000000 1052824.186499 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.923028] kdmwork-253:70 18384 177930165.020303 811212620 120 0.000000 26815472.393349 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.934364] bioset 18493 13398.023827 2 100 0.000000 0.010670 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.945726] agetty 51054 44473.296457 8 120 0.000000 6.734153 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.957072] ptlrpcd_00_07 64812 177929717.819748 4213658 120 0.000000 137769.359132 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.968422] ll_ost_create00 66925 177931075.789953 313682 120 0.000000 3723.956133 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.979757] ll_ost_io00_002 66951 177929615.394434 101725482 120 0.000000 5972625.845358 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592645.991101] ll_ost00_005 68146 169397267.958769 30213369 120 0.000000 663444.452739 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.002452] ll_ost00_010 68477 175571207.387214 38821098 120 0.000000 848317.667680 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.013784] ll_ost00_011 68479 173297977.323776 25515833 120 0.000000 564972.153451 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.025125] ll_ost_io00_018 68507 177929728.470835 113091565 120 0.000000 7803193.055926 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.036468] ll_ost00_013 68542 177930332.189889 35136112 120 0.000000 781852.078769 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.047816] ll_ost00_042 68662 169397267.714860 51513989 120 0.000000 1100908.718723 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.059154] ll_ost00_043 68664 169397267.758541 34407440 120 0.000000 761902.767570 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.070493] ll_ost00_049 68688 169397267.739305 56354879 120 0.000000 1232147.265741 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.081826] ll_ost_io00_020 68705 177929951.722417 87270408 120 0.000000 6296535.918154 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.093170] ll_ost00_063 68776 169397267.720129 53160152 120 0.000000 1136516.048130 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.104511] ll_ost00_074 68819 173577388.231110 34682318 120 0.000000 744737.482711 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.115853] ll_ost_create00 69326 177700025.593877 277774 120 0.000000 3448.152022 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.127223] ldlm_cn00_009 111304 177930443.198892 21226302 120 0.000000 502877.848429 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.138652] ldlm_cn00_021 30585 177930749.189389 36198503 120 0.000000 816898.834096 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.149990] ldlm_cn00_027 30591 177930441.993762 15702725 120 0.000000 334960.623680 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.161337] ll_ost_io00_053 87179 177930022.547819 114371753 120 0.000000 7033988.625713 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.172674] ll_ost_io00_057 87212 177930048.760457 102544964 120 0.000000 6654069.435590 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.184006] ll_ost_io00_058 87558 177930158.704506 62780721 120 0.000000 4208381.948132 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.195340] ll_ost_io00_060 87560 177929953.056042 143889623 120 0.000000 9310085.610437 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.206699] ll_ost_io00_099 40823 177929867.177325 111600538 120 0.000000 7445245.476894 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.218078] kworker/36:0 6468 177898942.331700 38057 120 0.000000 366.289347 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.229419] kworker/36:2 13435 177931076.339598 2743 120 0.000000 49.511146 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.240761] sleep 14312 177931075.630828 2 120 0.000000 0.544356 0.000000 0 / [Tue Mar 17 18:02:41 2020][8592646.252128] [Tue Mar 17 18:02:41 2020][8592646.253800] cpu#37, 1996.233 MHz [Tue Mar 17 18:02:41 2020][8592646.257208] .nr_running : 0 [Tue Mar 17 18:02:41 2020][8592646.262003] .load : 0 [Tue Mar 17 18:02:41 2020][8592646.266802] .nr_switches : 7819003105 [Tue Mar 17 18:02:41 2020][8592646.272377] .nr_load_updates : 2190484329 [Tue Mar 17 18:02:41 2020][8592646.277963] .nr_uninterruptible : 224534 [Tue Mar 17 18:02:41 2020][8592646.283192] .next_balance : 12887.290784 [Tue Mar 17 18:02:41 2020][8592646.288938] .curr->pid : 0 [Tue Mar 17 18:02:41 2020][8592646.293743] .clock : 8592646293.424362 [Tue Mar 17 18:02:41 2020][8592646.299928] .cpu_load[0] : 0 [Tue Mar 17 18:02:41 2020][8592646.304719] .cpu_load[1] : 0 [Tue Mar 17 18:02:41 2020][8592646.309528] .cpu_load[2] : 0 [Tue Mar 17 18:02:41 2020][8592646.314322] .cpu_load[3] : 0 [Tue Mar 17 18:02:41 2020][8592646.319116] .cpu_load[4] : 0 [Tue Mar 17 18:02:41 2020][8592646.323909] .avg_idle : 10000000 [Tue Mar 17 18:02:41 2020][8592646.329335] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:41 2020][8592646.334654] [Tue Mar 17 18:02:41 2020][8592646.336323] cfs_rq[37]:/ [Tue Mar 17 18:02:41 2020][8592646.339037] .exec_clock : 0.000000 [Tue Mar 17 18:02:41 2020][8592646.344435] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:41 2020][8592646.349834] .min_vruntime : 90387616.882078 [Tue Mar 17 18:02:41 2020][8592646.355839] .max_vruntime : 0.000001 [Tue Mar 17 18:02:41 2020][8592646.361241] .spread : 0.000000 [Tue Mar 17 18:02:41 2020][8592646.366639] .spread0 : -226827526.450673 [Tue Mar 17 18:02:41 2020][8592646.372818] .nr_spread_over : 0 [Tue Mar 17 18:02:41 2020][8592646.377611] .nr_running : 0 [Tue Mar 17 18:02:41 2020][8592646.382403] .load : 0 [Tue Mar 17 18:02:41 2020][8592646.387196] .runnable_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592646.391989] .blocked_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592646.396783] .tg_load_avg : 0 [Tue Mar 17 18:02:41 2020][8592646.401574] .tg_load_contrib : 0 [Tue Mar 17 18:02:41 2020][8592646.406368] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:41 2020][8592646.411159] .tg->runnable_avg : 0 [Tue Mar 17 18:02:41 2020][8592646.415952] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:41 2020][8592646.420746] .throttled : 0 [Tue Mar 17 18:02:41 2020][8592646.425537] .throttle_count : 0 [Tue Mar 17 18:02:41 2020][8592646.430332] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:41 2020][8592646.435124] .avg->runnable_avg_period : 48015 [Tue Mar 17 18:02:42 2020][8592646.440265] [Tue Mar 17 18:02:42 2020][8592646.441935] rt_rq[37]:/ [Tue Mar 17 18:02:42 2020][8592646.444561] .rt_nr_running : 0 [Tue Mar 17 18:02:42 2020][8592646.449354] .rt_throttled : 0 [Tue Mar 17 18:02:42 2020][8592646.454146] .rt_time : 0.000000 [Tue Mar 17 18:02:42 2020][8592646.459546] .rt_runtime : 950.000000 [Tue Mar 17 18:02:42 2020][8592646.465118] [Tue Mar 17 18:02:42 2020][8592646.466792] runnable tasks: [Tue Mar 17 18:02:42 2020][8592646.469765] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:42 2020][8592646.480711] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:42 2020][8592646.491601] watchdog/37 196 -11.984661 2148158 0 0.000000 15190.301083 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.502931] migration/37 197 0.000000 522436 0 0.000000 19832.314730 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.514267] ksoftirqd/37 198 90385989.426409 73616397 120 0.000000 918687.228324 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.525606] kworker/37:0H 200 8211.565099 10 100 0.000000 0.044693 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.536990] bioset 12568 6344.488625 2 100 0.000000 0.010850 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.548348] kworker/37:1H 14203 90385989.416333 130114783 100 0.000000 660575.069679 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.559733] master 51202 90387592.815240 1680688 120 0.000000 104437.986070 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.571084] ptlrpcd_01_00 64817 90387604.920081 4391927 120 0.000000 138679.792141 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.582425] bioset 65718 63899.072483 2 100 0.000000 0.006613 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.593769] ldlm_bl_01 66895 90387604.897468 6336392 120 0.000000 87221.854047 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.605115] ll_ost01_006 67861 89789831.015708 23401994 120 0.000000 622152.552799 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.616449] ldiskfs-rsv-con 67888 65453.857818 2 100 0.000000 0.012123 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.627785] inconsistency_v 68150 65838.051984 2 120 0.000000 0.031188 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.639122] inconsistency_v 68179 66062.487358 2 120 0.000000 0.022262 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.650457] inconsistency_v 68207 66270.363431 2 120 0.000000 0.021219 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.661793] inconsistency_v 68234 66469.897716 2 120 0.000000 0.022251 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.673129] inconsistency_v 68261 66666.579014 2 120 0.000000 0.023935 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.684466] inconsistency_v 68288 66861.853900 2 120 0.000000 0.021590 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.695824] ll_ost01_017 68538 87250035.743245 16593334 120 0.000000 431729.021049 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.707166] ll_ost01_035 68610 87250035.740540 18853534 120 0.000000 537912.081945 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.718504] ll_ost01_045 68645 72431341.823826 8830787 120 0.000000 209492.545647 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.729842] ll_ost01_060 68704 87250039.941182 21250279 120 0.000000 583981.881626 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.741179] ll_ost_io01_019 68706 90385852.465059 20580039 120 0.000000 1312758.950503 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.752538] ll_ost_create01 68904 89325149.595593 2640514 120 0.000000 46270.472933 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.763889] ll_ost01_087 97763 87250035.737385 25520498 120 0.000000 628749.637235 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.775233] ldlm_bl_21 100102 90387604.907466 6106274 120 0.000000 83529.932447 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.786721] ll_ost01_097 101307 72431341.823135 11276276 120 0.000000 289839.367554 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.798140] ll_ost01_107 101319 87250035.700194 10211999 120 0.000000 259695.886438 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.809598] kworker/37:1 10852 90299557.467041 91 120 0.000000 0.260523 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.820932] kworker/37:2 11979 90387604.891778 6181 120 0.000000 105.962896 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.832276] R sleep 14254 90387604.998327 2 120 0.000000 0.787683 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.843613] kworker/u256:0 14295 90387604.992505 47 120 0.000000 0.212529 0.000000 1 / [Tue Mar 17 18:02:42 2020][8592646.854988] [Tue Mar 17 18:02:42 2020][8592646.856660] cpu#38, 1996.233 MHz [Tue Mar 17 18:02:42 2020][8592646.860066] .nr_running : 0 [Tue Mar 17 18:02:42 2020][8592646.864858] .load : 0 [Tue Mar 17 18:02:42 2020][8592646.869652] .nr_switches : 13389799368 [Tue Mar 17 18:02:42 2020][8592646.875327] .nr_load_updates : 3014513215 [Tue Mar 17 18:02:42 2020][8592646.880902] .nr_uninterruptible : -15523626 [Tue Mar 17 18:02:42 2020][8592646.886388] .next_balance : 12887.291386 [Tue Mar 17 18:02:42 2020][8592646.892147] .curr->pid : 0 [Tue Mar 17 18:02:42 2020][8592646.896943] .clock : 8592646896.422349 [Tue Mar 17 18:02:42 2020][8592646.903121] .cpu_load[0] : 0 [Tue Mar 17 18:02:42 2020][8592646.907928] .cpu_load[1] : 0 [Tue Mar 17 18:02:42 2020][8592646.912726] .cpu_load[2] : 0 [Tue Mar 17 18:02:42 2020][8592646.917518] .cpu_load[3] : 0 [Tue Mar 17 18:02:42 2020][8592646.922311] .cpu_load[4] : 2 [Tue Mar 17 18:02:42 2020][8592646.927131] .avg_idle : 3647367 [Tue Mar 17 18:02:42 2020][8592646.932451] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:42 2020][8592646.937764] [Tue Mar 17 18:02:42 2020][8592646.939431] cfs_rq[38]:/ [Tue Mar 17 18:02:42 2020][8592646.942145] .exec_clock : 0.000000 [Tue Mar 17 18:02:42 2020][8592646.947544] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:42 2020][8592646.952944] .min_vruntime : 152887198.772763 [Tue Mar 17 18:02:42 2020][8592646.959037] .max_vruntime : 0.000001 [Tue Mar 17 18:02:42 2020][8592646.964437] .spread : 0.000000 [Tue Mar 17 18:02:42 2020][8592646.969836] .spread0 : -164327944.559988 [Tue Mar 17 18:02:42 2020][8592646.976013] .nr_spread_over : 0 [Tue Mar 17 18:02:42 2020][8592646.980806] .nr_running : 0 [Tue Mar 17 18:02:42 2020][8592646.985599] .load : 0 [Tue Mar 17 18:02:42 2020][8592646.990393] .runnable_load_avg : 0 [Tue Mar 17 18:02:42 2020][8592646.995186] .blocked_load_avg : 0 [Tue Mar 17 18:02:42 2020][8592646.999978] .tg_load_avg : 0 [Tue Mar 17 18:02:42 2020][8592647.004772] .tg_load_contrib : 0 [Tue Mar 17 18:02:42 2020][8592647.009562] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:42 2020][8592647.014357] .tg->runnable_avg : 0 [Tue Mar 17 18:02:42 2020][8592647.019148] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:42 2020][8592647.023941] .throttled : 0 [Tue Mar 17 18:02:42 2020][8592647.028735] .throttle_count : 0 [Tue Mar 17 18:02:42 2020][8592647.033528] .avg->runnable_avg_sum : 229 [Tue Mar 17 18:02:42 2020][8592647.038491] .avg->runnable_avg_period : 46811 [Tue Mar 17 18:02:42 2020][8592647.043635] [Tue Mar 17 18:02:42 2020][8592647.045306] rt_rq[38]:/ [Tue Mar 17 18:02:42 2020][8592647.047931] .rt_nr_running : 0 [Tue Mar 17 18:02:42 2020][8592647.052724] .rt_throttled : 0 [Tue Mar 17 18:02:42 2020][8592647.057517] .rt_time : 0.000000 [Tue Mar 17 18:02:42 2020][8592647.062914] .rt_runtime : 950.000000 [Tue Mar 17 18:02:42 2020][8592647.068487] [Tue Mar 17 18:02:42 2020][8592647.070160] runnable tasks: [Tue Mar 17 18:02:42 2020][8592647.073134] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:42 2020][8592647.084080] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:42 2020][8592647.094938] systemd 1 152887148.978284 3414547 120 0.000000 435181.419740 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.106308] watchdog/38 201 -11.987026 2148160 0 0.000000 20915.952889 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.117645] migration/38 202 0.000000 543033 0 0.000000 185479.538022 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.128982] ksoftirqd/38 203 152887183.966752 568059957 120 0.000000 5775111.326774 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.140318] kworker/38:0H 205 116531585.488945 20 100 0.000000 0.219002 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.151669] scsi_eh_1 1958 657.502969 2 120 0.000000 0.011221 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.163009] mlx5_events 4715 1397.883968 2 100 0.000000 0.011191 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.174344] mlx5_fw_tracer 4717 1421.898981 2 100 0.000000 0.009218 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.185678] mlx5_fc 4719 1445.947890 2 100 0.000000 0.009237 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.197016] poll_mpt3sas0_h 4721 1457.956786 2 100 0.000000 0.009567 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.208353] ib-comp-wq 4743 1590.058285 2 100 0.000000 0.009579 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.219688] kworker/u261:0 4744 1638.088260 3 100 0.000000 0.011171 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.231025] kworker/u264:0 4747 1638.088257 3 100 0.000000 0.011091 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.242360] ib-comp-unb-wq 4749 1662.096473 2 100 0.000000 0.008897 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.253698] mlx5_ib_sigerr_ 4763 1699.581607 2 100 0.000000 0.012463 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.265032] ib_mad1 4765 1711.592032 2 100 0.000000 0.011432 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.276367] to_fifo 4766 1723.601923 2 100 0.000000 0.010709 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.287703] mkey_cache 4767 1735.610919 2 100 0.000000 0.009637 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.299042] poll_mpt3sas0_s 4777 1843.967316 2 100 0.000000 0.010520 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.310381] ext4-rsv-conver 5657 3235.012878 2 100 0.000000 0.010430 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.321716] ccp-4-q0 6479 3781.475522 2 120 0.000000 0.009988 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.333048] ccp-4-q1 6480 3793.483761 2 120 0.000000 0.008245 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.344384] ccp-4-q2 6481 3805.492202 2 120 0.000000 0.008447 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.355720] ccp-4-q3 6482 3817.500962 2 120 0.000000 0.008766 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.367057] ccp-4-q4 6483 3829.508991 2 120 0.000000 0.008035 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.378394] ccp-6-q4 6491 3841.518653 2 120 0.000000 0.009668 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.389819] rpciod 50143 34464.818181 2 100 0.000000 0.010179 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.401151] xprtiod 50144 34476.826888 2 100 0.000000 0.009378 0.000000 2 / [Tue Mar 17 18:02:42 2020][8592647.412489] gdbus 50223 152887096.859752 2570228 120 0.000000 141107.615331 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.423824] polkitd 50225 34610.134300 1 120 0.000000 0.048120 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.435159] JS GC Helper 50227 34634.357098 5 120 0.000000 0.102042 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.446496] JS Sour~ Thread 50229 34658.905272 5 120 0.000000 0.078688 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.457833] gssproxy 50203 34585.901374 3 120 0.000000 4.724628 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.469170] mlx4 50359 34916.219801 2 100 0.000000 0.013545 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.480505] mlx4_ib 50367 34940.407363 2 100 0.000000 0.012904 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.491840] mlx4_ib_mcg 50368 34952.416931 2 100 0.000000 0.010259 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.503176] mlx5e 50419 35038.165752 2 100 0.000000 0.013435 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.514512] ipoib_wq 50420 35050.177356 2 100 0.000000 0.012433 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.525849] rdma_cm 50439 35086.597040 2 100 0.000000 0.013226 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.537185] in:imjournal 50889 152887198.772763 10336483 120 0.000000 359034.834403 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.548520] collectd 50885 152887157.877300 1044021 120 0.000000 71042.595002 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.559858] nfsiod 50940 36029.567006 2 100 0.000000 0.010109 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.571194] nfsv4.1-svc 50968 139231220.307703 8 120 0.000000 0.144885 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.582535] ptlrpc_hr02_000 64760 62094.714166 2 120 0.000000 0.024516 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.593865] ptlrpc_hr02_002 64762 62106.733235 2 120 0.000000 0.019075 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.605200] ptlrpc_hr02_003 64763 62118.745421 2 120 0.000000 0.012192 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.616536] ptlrpc_hr02_004 64764 62130.758680 2 120 0.000000 0.013265 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.627874] ptlrpc_hr02_005 64765 62142.772460 2 120 0.000000 0.013786 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.639221] ptlrpcd_02_05 64834 152885966.658236 2557087 120 0.000000 99107.393810 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.650558] bioset 65623 67887.718806 2 100 0.000000 0.011562 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.661891] bioset 65624 67899.727872 2 100 0.000000 0.009578 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.673229] bioset 66217 68462.416229 2 100 0.000000 0.010339 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.684565] jbd2/md8-8 66764 152885981.217732 15412458 120 0.000000 2887085.657794 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.695906] ll_ost02_002 66920 152886502.612154 46753586 120 0.000000 1346724.552207 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.707245] ll_ost_out02_00 67048 115164006.717653 27 120 0.000000 1.368099 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.718594] ll_ost_io02_003 68348 152886290.781848 88320187 120 0.000000 6551143.824477 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.729928] ll_ost_io02_008 68375 152885967.805100 101743581 120 0.000000 6664183.208839 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.741262] ll_ost_io02_009 68381 152886149.127486 143129963 120 0.000000 10505617.937536 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.752602] ll_ost02_014 68474 149723162.553394 44750316 120 0.000000 1290010.882573 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.763932] ll_ost02_015 68475 149576479.818874 37207663 120 0.000000 1063644.209091 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.775271] ll_ost02_018 68482 152886156.701406 42233082 120 0.000000 1203554.429534 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.786606] ll_ost_io02_013 68485 152885986.966599 56426216 120 0.000000 3404247.170474 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.797943] ll_ost_io02_016 68495 152886207.827695 174202465 120 0.000000 11158934.354426 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.809288] ll_ost02_027 68561 152886334.408899 30975125 120 0.000000 928602.134763 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.820636] ll_ost02_050 68667 149723162.553694 34532174 120 0.000000 980310.332524 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.831979] ll_ost_io02_022 68707 152885947.316302 99592338 120 0.000000 7008422.714576 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.843311] ll_ost_io02_023 68719 152885968.924457 173001958 120 0.000000 10978173.238451 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.854654] ll_ost02_068 68771 149576479.818283 26137036 120 0.000000 770292.530759 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.865991] ll_ost_io02_029 68775 152886161.749372 130219376 120 0.000000 8655191.056912 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.877346] ll_ost_create02 69477 152887186.777822 3367781 120 0.000000 63739.252329 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.888696] ll_ost_create02 79128 151507485.479756 4102539 120 0.000000 77890.991179 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.900036] ll_ost_io02_052 97760 152886137.242369 125334510 120 0.000000 8355193.398761 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.911376] ll_ost02_097 98688 152887187.566814 48232588 120 0.000000 1380430.109985 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.922728] ldlm_cn02_012 111317 152885610.861263 25409374 120 0.000000 620270.365941 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.934147] ldlm_cn02_013 111319 152886911.039838 19189070 120 0.000000 448890.207135 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.945582] ldlm_cn02_029 64122 152885610.748078 17499402 120 0.000000 402838.947036 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.956928] ll_ost_io02_071 90840 152885933.598336 126213623 120 0.000000 7984570.982255 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.968264] ll_ost_io02_081 90856 152886150.853970 48736802 120 0.000000 3248796.122166 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.979597] ll_ost_io02_082 90859 152886037.682772 81462645 120 0.000000 5430174.856159 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592647.990934] ll_ost_io02_085 90867 152886159.543981 95889674 120 0.000000 6871230.852597 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.002267] ll_ost_io02_086 90870 152885980.632135 133735389 120 0.000000 8820798.052284 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.013618] ll_ost_io02_101 40798 152886018.687303 79012132 120 0.000000 5286700.120726 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.024956] ll_ost_io02_102 40801 152885975.387187 143872986 120 0.000000 9544554.138567 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.036294] ll_ost_io02_103 40803 152886227.482103 105629229 120 0.000000 7059877.564160 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.047638] ll_ost_io02_109 40835 152885952.848402 85796383 120 0.000000 5858822.138385 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.058986] ll_ost02_103 101331 152886354.357452 40638366 120 0.000000 1178683.939736 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.070416] ll_ost02_111 85266 152886010.059835 25566219 120 0.000000 698981.321342 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.081757] kworker/38:2H 10254 152886930.896478 373759661 100 0.000000 1923296.770270 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.093111] kworker/38:0 11947 152887186.779045 15072 120 0.000000 259.358692 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.104451] kworker/u259:3 13342 152886931.018239 902 120 0.000000 24.162292 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.115788] kworker/38:1 13980 152883627.767556 3 120 0.000000 0.016553 0.000000 2 / [Tue Mar 17 18:02:43 2020][8592648.127149] [Tue Mar 17 18:02:43 2020][8592648.128820] cpu#39, 1996.233 MHz [Tue Mar 17 18:02:43 2020][8592648.132229] .nr_running : 0 [Tue Mar 17 18:02:43 2020][8592648.137023] .load : 0 [Tue Mar 17 18:02:43 2020][8592648.141814] .nr_switches : 9736035970 [Tue Mar 17 18:02:43 2020][8592648.147389] .nr_load_updates : 2572281819 [Tue Mar 17 18:02:43 2020][8592648.152974] .nr_uninterruptible : 248227 [Tue Mar 17 18:02:43 2020][8592648.158202] .next_balance : 12887.292658 [Tue Mar 17 18:02:43 2020][8592648.163949] .curr->pid : 0 [Tue Mar 17 18:02:43 2020][8592648.168758] .clock : 8592648168.427270 [Tue Mar 17 18:02:43 2020][8592648.174938] .cpu_load[0] : 0 [Tue Mar 17 18:02:43 2020][8592648.179730] .cpu_load[1] : 0 [Tue Mar 17 18:02:43 2020][8592648.184538] .cpu_load[2] : 0 [Tue Mar 17 18:02:43 2020][8592648.189335] .cpu_load[3] : 0 [Tue Mar 17 18:02:43 2020][8592648.194127] .cpu_load[4] : 0 [Tue Mar 17 18:02:43 2020][8592648.198920] .avg_idle : 10000000 [Tue Mar 17 18:02:43 2020][8592648.204333] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:43 2020][8592648.209648] [Tue Mar 17 18:02:43 2020][8592648.211318] cfs_rq[39]:/ [Tue Mar 17 18:02:43 2020][8592648.214030] .exec_clock : 0.000000 [Tue Mar 17 18:02:43 2020][8592648.219432] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:43 2020][8592648.224829] .min_vruntime : 100430080.672173 [Tue Mar 17 18:02:43 2020][8592648.230923] .max_vruntime : 0.000001 [Tue Mar 17 18:02:43 2020][8592648.236322] .spread : 0.000000 [Tue Mar 17 18:02:43 2020][8592648.241721] .spread0 : -216785062.660578 [Tue Mar 17 18:02:43 2020][8592648.247900] .nr_spread_over : 0 [Tue Mar 17 18:02:43 2020][8592648.252694] .nr_running : 0 [Tue Mar 17 18:02:43 2020][8592648.257485] .load : 0 [Tue Mar 17 18:02:43 2020][8592648.262279] .runnable_load_avg : 0 [Tue Mar 17 18:02:43 2020][8592648.267072] .blocked_load_avg : 0 [Tue Mar 17 18:02:43 2020][8592648.271864] .tg_load_avg : 0 [Tue Mar 17 18:02:43 2020][8592648.276657] .tg_load_contrib : 0 [Tue Mar 17 18:02:43 2020][8592648.281448] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:43 2020][8592648.286241] .tg->runnable_avg : 0 [Tue Mar 17 18:02:43 2020][8592648.291033] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:43 2020][8592648.295828] .throttled : 0 [Tue Mar 17 18:02:43 2020][8592648.300619] .throttle_count : 0 [Tue Mar 17 18:02:43 2020][8592648.305412] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:43 2020][8592648.310206] .avg->runnable_avg_period : 46477 [Tue Mar 17 18:02:43 2020][8592648.315347] [Tue Mar 17 18:02:43 2020][8592648.317016] rt_rq[39]:/ [Tue Mar 17 18:02:43 2020][8592648.319644] .rt_nr_running : 0 [Tue Mar 17 18:02:43 2020][8592648.324436] .rt_throttled : 0 [Tue Mar 17 18:02:43 2020][8592648.329228] .rt_time : 0.000000 [Tue Mar 17 18:02:43 2020][8592648.334628] .rt_runtime : 950.000000 [Tue Mar 17 18:02:43 2020][8592648.340201] [Tue Mar 17 18:02:43 2020][8592648.341872] runnable tasks: [Tue Mar 17 18:02:43 2020][8592648.344847] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:43 2020][8592648.355791] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:43 2020][8592648.366684] watchdog/39 206 -11.984892 2148159 0 0.000000 14152.228833 0.000000 3 / [Tue Mar 17 18:02:43 2020][8592648.378021] migration/39 207 0.000000 666235 0 0.000000 34207.287817 0.000000 3 / [Tue Mar 17 18:02:43 2020][8592648.389358] ksoftirqd/39 208 100428614.523644 102199984 120 0.000000 1192939.970103 0.000000 3 / [Tue Mar 17 18:02:43 2020][8592648.400694] kworker/39:0H 210 2123.668001 10 100 0.000000 0.032410 0.000000 3 / [Tue Mar 17 18:02:43 2020][8592648.412046] ib_mcast 4750 1138.634792 2 100 0.000000 0.009347 0.000000 3 / [Tue Mar 17 18:02:43 2020][8592648.423384] ib_nl_sa_wq 4751 1150.642637 2 100 0.000000 0.008595 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.434724] kworker/39:1H 5590 100428614.438383 162387069 100 0.000000 719366.763860 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.446062] ccp-2-q0 6431 2806.911612 2 120 0.000000 0.011021 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.457401] ccp-2-q1 6433 2818.925502 2 120 0.000000 0.013896 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.468736] ccp-2-q2 6434 2830.936186 2 120 0.000000 0.010690 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.480072] ccp-2-q3 6435 2842.945789 2 120 0.000000 0.009609 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.491409] ccp-2-q4 6436 2854.954850 2 120 0.000000 0.009067 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.502747] ccp-6-q0 6487 2882.158943 2 120 0.000000 0.010530 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.514082] ccp-6-q1 6488 2894.167743 2 120 0.000000 0.008806 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.525416] ccp-6-q2 6489 2906.175852 2 120 0.000000 0.008115 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.536754] ccp-6-q3 6490 2918.184593 2 120 0.000000 0.008747 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.548169] kdmwork-253:104 20039 443233.341726 74122 120 0.000000 908.760684 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.559519] reader#4 50903 28029.826930 1 120 0.000000 0.010670 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.570874] ptlrpcd_03_04 64845 100428045.515584 3447127 120 0.000000 119945.252272 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.582228] ll_ost_out03_00 67049 77804505.186956 1507873 120 0.000000 32772.397688 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.593575] ll_ost03_006 68338 100428614.553089 16640864 120 0.000000 459280.510547 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.604919] ll_ost_io03_010 68395 100428253.956866 55480959 120 0.000000 3553835.558731 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.616253] ll_ost_io03_012 68398 100428585.511396 101052197 120 0.000000 6108233.888644 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.627599] ll_ost_io03_016 68499 100428507.045427 80154390 120 0.000000 4303851.678796 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.638964] ll_ost03_062 68744 100428615.407749 15239352 120 0.000000 467140.064741 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.650303] ll_ost03_066 68799 98802460.052234 14740615 120 0.000000 436744.200535 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.661654] ldlm_cn03_005 69307 100103404.653010 20662397 120 0.000000 472928.435943 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.672987] ldlm_cn03_006 70323 100427664.021632 14869186 120 0.000000 316705.951994 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.684325] ldlm_cn03_007 72311 100427685.665698 22400395 120 0.000000 521147.771842 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.695667] ll_ost_create03 75352 100230276.369653 3433876 120 0.000000 58099.509396 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.707040] ldlm_cn03_017 31075 100429797.973375 11338818 120 0.000000 247161.625386 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.718373] ldlm_cn03_018 31077 100427130.961309 20600778 120 0.000000 478735.688513 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.729714] ldlm_cn03_020 64126 100427130.961370 26224926 120 0.000000 606692.262127 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.741044] ldlm_cn03_021 64127 100427130.961599 19606185 120 0.000000 431995.477356 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.752383] ldlm_cn03_026 64132 100427664.021492 21985688 120 0.000000 511923.238511 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.763721] ll_ost_io03_058 87196 100428620.455118 166719972 120 0.000000 9909915.297429 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.775055] ll_ost_io03_060 87198 100428614.538602 76830577 120 0.000000 5115333.233589 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.786391] ll_ost_io03_062 87202 100428482.626217 97437810 120 0.000000 6459374.049984 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.797736] ll_ost_io03_073 90872 100428213.102946 109473877 120 0.000000 6524514.094610 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.809087] ll_ost_io03_087 40810 100428497.634040 125906343 120 0.000000 7241033.353099 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.820432] ll_ost_io03_100 40847 100428536.972206 98218055 120 0.000000 5443684.278080 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.831772] ll_ost_io03_103 40854 100428210.250554 139212872 120 0.000000 9110419.882192 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.843104] ll_ost_io03_105 40858 100428046.975592 120423377 120 0.000000 7302195.989966 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.854464] ll_ost03_105 125738 98802460.053205 8483884 120 0.000000 268161.381335 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.865897] kworker/39:0 4805 100430068.712489 41431 120 0.000000 685.069032 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.877235] kworker/39:1 8167 100224615.287218 2 120 0.000000 0.010971 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.888581] sleep 14266 100429940.175777 2 120 0.000000 0.541530 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.899917] sleep 14320 100430067.793805 2 120 0.000000 0.642940 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.911251] sleep 14327 100430080.672173 2 120 0.000000 0.667337 0.000000 3 / [Tue Mar 17 18:02:44 2020][8592648.922629] [Tue Mar 17 18:02:44 2020][8592648.924306] cpu#40, 1996.233 MHz [Tue Mar 17 18:02:44 2020][8592648.927713] .nr_running : 0 [Tue Mar 17 18:02:44 2020][8592648.932507] .load : 0 [Tue Mar 17 18:02:44 2020][8592648.937301] .nr_switches : 9717650060 [Tue Mar 17 18:02:44 2020][8592648.942882] .nr_load_updates : 2438869077 [Tue Mar 17 18:02:44 2020][8592648.948481] .nr_uninterruptible : 1039941 [Tue Mar 17 18:02:44 2020][8592648.953802] .next_balance : 12887.293455 [Tue Mar 17 18:02:44 2020][8592648.959547] .curr->pid : 0 [Tue Mar 17 18:02:44 2020][8592648.964354] .clock : 8592648963.431451 [Tue Mar 17 18:02:44 2020][8592648.970536] .cpu_load[0] : 0 [Tue Mar 17 18:02:44 2020][8592648.975328] .cpu_load[1] : 0 [Tue Mar 17 18:02:44 2020][8592648.980122] .cpu_load[2] : 0 [Tue Mar 17 18:02:44 2020][8592648.984929] .cpu_load[3] : 0 [Tue Mar 17 18:02:44 2020][8592648.989723] .cpu_load[4] : 0 [Tue Mar 17 18:02:44 2020][8592648.994517] .avg_idle : 10000000 [Tue Mar 17 18:02:44 2020][8592648.999943] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:44 2020][8592649.005254] [Tue Mar 17 18:02:44 2020][8592649.006924] cfs_rq[40]:/ [Tue Mar 17 18:02:44 2020][8592649.009636] .exec_clock : 0.000000 [Tue Mar 17 18:02:44 2020][8592649.015037] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:44 2020][8592649.020437] .min_vruntime : 169078337.253399 [Tue Mar 17 18:02:44 2020][8592649.026529] .max_vruntime : 0.000001 [Tue Mar 17 18:02:44 2020][8592649.031926] .spread : 0.000000 [Tue Mar 17 18:02:44 2020][8592649.037325] .spread0 : -148136806.079352 [Tue Mar 17 18:02:44 2020][8592649.043506] .nr_spread_over : 0 [Tue Mar 17 18:02:44 2020][8592649.048298] .nr_running : 0 [Tue Mar 17 18:02:44 2020][8592649.053092] .load : 0 [Tue Mar 17 18:02:44 2020][8592649.057885] .runnable_load_avg : 0 [Tue Mar 17 18:02:44 2020][8592649.062677] .blocked_load_avg : 0 [Tue Mar 17 18:02:44 2020][8592649.067469] .tg_load_avg : 0 [Tue Mar 17 18:02:44 2020][8592649.072261] .tg_load_contrib : 0 [Tue Mar 17 18:02:44 2020][8592649.077055] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:44 2020][8592649.081848] .tg->runnable_avg : 0 [Tue Mar 17 18:02:44 2020][8592649.086641] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:44 2020][8592649.091433] .throttled : 0 [Tue Mar 17 18:02:44 2020][8592649.096226] .throttle_count : 0 [Tue Mar 17 18:02:44 2020][8592649.101019] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:44 2020][8592649.105812] .avg->runnable_avg_period : 47151 [Tue Mar 17 18:02:44 2020][8592649.110952] [Tue Mar 17 18:02:44 2020][8592649.112623] rt_rq[40]:/ [Tue Mar 17 18:02:44 2020][8592649.115248] .rt_nr_running : 0 [Tue Mar 17 18:02:44 2020][8592649.120042] .rt_throttled : 0 [Tue Mar 17 18:02:44 2020][8592649.124834] .rt_time : 0.000000 [Tue Mar 17 18:02:44 2020][8592649.130233] .rt_runtime : 950.000000 [Tue Mar 17 18:02:44 2020][8592649.135807] [Tue Mar 17 18:02:44 2020][8592649.137478] runnable tasks: [Tue Mar 17 18:02:44 2020][8592649.140450] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:44 2020][8592649.151398] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:44 2020][8592649.162291] watchdog/40 211 -11.988879 2148159 0 0.000000 16294.374738 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.173627] migration/40 212 0.000000 71824 0 0.000000 41937.120491 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.184965] ksoftirqd/40 213 169078281.393338 238619783 120 0.000000 2464571.674599 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.196299] kworker/40:0H 215 16745.840847 10 100 0.000000 0.035078 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.207661] ccp-7-q2 6492 1906.270788 2 120 0.000000 0.009619 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.218999] ccp-7-q3 6493 1918.279920 2 120 0.000000 0.009138 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.230335] ccp-7-q4 6494 1930.288660 2 120 0.000000 0.008746 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.241671] ccp-8-q0 6495 1942.298232 2 120 0.000000 0.009578 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.253006] ccp-8-q1 6496 1954.306982 2 120 0.000000 0.008756 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.264342] ccp-8-q2 6497 1966.315712 2 120 0.000000 0.008736 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.275678] ccp-8-q3 6498 1978.324041 2 120 0.000000 0.008335 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.287015] ccp-8-q4 6499 1990.332751 2 120 0.000000 0.008716 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.298376] kdmwork-253:6 12517 488549.695707 222370 120 0.000000 3134.804964 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.309773] kworker/40:1H 25524 169078257.933479 355365901 100 0.000000 1393185.157999 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.321114] bioset 30890 19732.254809 2 100 0.000000 0.010109 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.332452] writer#3 50897 169078324.061983 49054977 120 0.000000 722675.106746 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.343805] ptlrpcd_00_11 64816 169077960.000172 4204919 120 0.000000 137649.190648 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.355148] jbd2/md6-8 66371 169077959.929498 15307873 120 0.000000 2962311.967520 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.366501] ll_ost00_007 68339 166300737.047646 58238684 120 0.000000 1238894.014596 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.377840] ll_ost00_009 68376 167383557.050761 68009652 120 0.000000 1454583.901516 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.389181] ll_ost_io00_010 68383 169077971.925551 81408003 120 0.000000 5554545.435646 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.400516] ll_ost_io00_012 68388 169077990.844136 50857184 120 0.000000 3788141.540938 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.411853] ldlm_cn00_003 68444 169077590.340068 21710777 120 0.000000 521234.700203 0.000000 0 / [Tue Mar 17 18:02:44 2020][8592649.423194] ll_ost_io00_017 68494 169077953.532472 80316188 120 0.000000 5294992.752832 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.434544] ll_ost00_026 68591 169078337.253399 44021747 120 0.000000 1003114.060715 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.445880] ll_ost00_033 68624 169078264.793469 52101958 120 0.000000 1201076.714192 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.457216] ll_ost00_037 68648 164345952.904249 37107843 120 0.000000 802885.122783 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.468550] ll_ost00_040 68657 169078315.808256 76728525 120 0.000000 1709976.051110 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.479889] ll_ost00_052 68696 164345952.918355 29887074 120 0.000000 661704.595572 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.491223] ll_ost00_053 68711 164345952.896384 63199046 120 0.000000 1367356.868742 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.502571] ll_ost00_077 68823 169078311.144880 58811371 120 0.000000 1225917.602153 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.513925] ldlm_cn00_006 69929 169077590.698492 9652371 120 0.000000 227513.947560 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.525299] ldlm_cn00_012 111309 169077593.283730 26325362 120 0.000000 619451.094186 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.536725] ldlm_cn00_015 111322 169077595.423375 11275515 120 0.000000 261935.572738 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.548145] ldlm_cn00_018 30582 169077590.133507 24352318 120 0.000000 560086.189137 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.559481] ldlm_cn00_023 30587 169077590.136332 27302132 120 0.000000 624910.728696 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.570816] ldlm_cn00_024 30588 169077590.134449 11931252 120 0.000000 275315.297270 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.582164] ll_ost_io00_054 87192 169077959.892044 60035043 120 0.000000 4135115.027834 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.593501] ll_ost_io00_063 87563 169078261.110112 76908993 120 0.000000 5549113.153627 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.604840] ll_ost_io00_069 90868 169077974.428098 125505240 120 0.000000 8858799.087346 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.616179] ll_ost_io00_073 10668 169078137.197043 80309298 120 0.000000 5221796.395765 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.627517] ll_ost_io00_084 40239 169078148.067848 91008453 120 0.000000 6269065.092019 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.638859] ll_ost_io00_093 40793 169078259.021598 138861760 120 0.000000 8816979.500311 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.650212] ll_ost00_080 101318 164345952.900841 35598810 120 0.000000 788703.158943 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.661636] ll_ost00_081 101321 166300737.047545 20611685 120 0.000000 431559.858485 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.673068] ll_ost00_096 85262 164345952.894630 18290778 120 0.000000 429504.390495 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.684403] ll_ost00_101 85270 169078320.600403 27907692 120 0.000000 619514.082507 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.695752] kworker/40:2 48501 168599988.964615 9 120 0.000000 0.064421 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.707084] kworker/40:1 118397 169078324.748379 45133 120 0.000000 912.368627 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.718520] kworker/u262:2 13690 169078265.214748 835 100 0.000000 9.433456 0.000000 0 / [Tue Mar 17 18:02:45 2020][8592649.729911] [Tue Mar 17 18:02:45 2020][8592649.731585] cpu#41, 1996.233 MHz [Tue Mar 17 18:02:45 2020][8592649.734992] .nr_running : 0 [Tue Mar 17 18:02:45 2020][8592649.739788] .load : 0 [Tue Mar 17 18:02:45 2020][8592649.744588] .nr_switches : 8483968096 [Tue Mar 17 18:02:45 2020][8592649.750172] .nr_load_updates : 2285545840 [Tue Mar 17 18:02:45 2020][8592649.755751] .nr_uninterruptible : 200929 [Tue Mar 17 18:02:45 2020][8592649.760976] .next_balance : 12887.294263 [Tue Mar 17 18:02:45 2020][8592649.766737] .curr->pid : 0 [Tue Mar 17 18:02:45 2020][8592649.771534] .clock : 8592649771.432150 [Tue Mar 17 18:02:45 2020][8592649.777710] .cpu_load[0] : 0 [Tue Mar 17 18:02:45 2020][8592649.782518] .cpu_load[1] : 0 [Tue Mar 17 18:02:45 2020][8592649.787313] .cpu_load[2] : 0 [Tue Mar 17 18:02:45 2020][8592649.792107] .cpu_load[3] : 0 [Tue Mar 17 18:02:45 2020][8592649.796899] .cpu_load[4] : 0 [Tue Mar 17 18:02:45 2020][8592649.801707] .avg_idle : 10000000 [Tue Mar 17 18:02:45 2020][8592649.807108] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:45 2020][8592649.812421] [Tue Mar 17 18:02:45 2020][8592649.814092] cfs_rq[41]:/ [Tue Mar 17 18:02:45 2020][8592649.816804] .exec_clock : 0.000000 [Tue Mar 17 18:02:45 2020][8592649.822204] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:45 2020][8592649.827601] .min_vruntime : 79504454.059567 [Tue Mar 17 18:02:45 2020][8592649.833608] .max_vruntime : 0.000001 [Tue Mar 17 18:02:45 2020][8592649.839007] .spread : 0.000000 [Tue Mar 17 18:02:45 2020][8592649.844408] .spread0 : -237710689.273184 [Tue Mar 17 18:02:45 2020][8592649.850587] .nr_spread_over : 0 [Tue Mar 17 18:02:45 2020][8592649.855380] .nr_running : 0 [Tue Mar 17 18:02:45 2020][8592649.860171] .load : 0 [Tue Mar 17 18:02:45 2020][8592649.864964] .runnable_load_avg : 0 [Tue Mar 17 18:02:45 2020][8592649.869757] .blocked_load_avg : 0 [Tue Mar 17 18:02:45 2020][8592649.874550] .tg_load_avg : 0 [Tue Mar 17 18:02:45 2020][8592649.879344] .tg_load_contrib : 0 [Tue Mar 17 18:02:45 2020][8592649.884134] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:45 2020][8592649.888928] .tg->runnable_avg : 0 [Tue Mar 17 18:02:45 2020][8592649.893720] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:45 2020][8592649.898514] .throttled : 0 [Tue Mar 17 18:02:45 2020][8592649.903306] .throttle_count : 0 [Tue Mar 17 18:02:45 2020][8592649.908098] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:45 2020][8592649.912891] .avg->runnable_avg_period : 46627 [Tue Mar 17 18:02:45 2020][8592649.918034] [Tue Mar 17 18:02:45 2020][8592649.919705] rt_rq[41]:/ [Tue Mar 17 18:02:45 2020][8592649.922328] .rt_nr_running : 0 [Tue Mar 17 18:02:45 2020][8592649.927121] .rt_throttled : 0 [Tue Mar 17 18:02:45 2020][8592649.931915] .rt_time : 0.000000 [Tue Mar 17 18:02:45 2020][8592649.937315] .rt_runtime : 950.000000 [Tue Mar 17 18:02:45 2020][8592649.942886] [Tue Mar 17 18:02:45 2020][8592649.944560] runnable tasks: [Tue Mar 17 18:02:45 2020][8592649.947532] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:45 2020][8592649.958478] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:45 2020][8592649.969373] watchdog/41 216 -11.984301 2148159 0 0.000000 15058.872928 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592649.980709] migration/41 217 0.000000 37261 0 0.000000 16153.483893 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592649.992043] ksoftirqd/41 218 79504319.627947 70762135 120 0.000000 835731.077299 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.003379] kworker/41:0H 220 9109.917880 10 100 0.000000 0.039225 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.014727] scsi_eh_0 579 46.649802 2 120 0.000000 0.010079 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.026123] kworker/41:1H 18016 79504407.134494 126731188 100 0.000000 645180.167712 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.037479] kdmwork-253:86 18574 428785.011730 73806 120 0.000000 880.718472 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.048815] kdmwork-253:93 18815 79504419.134454 804369520 120 0.000000 29051372.729437 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.060166] auditd 50139 79504436.394325 1592866 116 0.000000 44132.942588 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.071521] ptlrpcd_01_05 64822 79504436.439657 4549525 120 0.000000 136494.933926 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.082870] ldlm_cn01_002 66888 79503398.161573 16498029 120 0.000000 355784.424523 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.094211] ll_ost01_002 66910 77191229.157732 24223037 120 0.000000 663243.516681 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.105560] ll_ost01_009 68347 79503741.139613 19029658 120 0.000000 454397.900722 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.116913] ll_ost_io01_014 68496 79503930.044793 99352894 120 0.000000 6688259.632507 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.128262] ll_ost01_025 68573 77191229.135351 23055450 120 0.000000 588388.197844 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.139596] ll_ost01_026 68575 79504443.110715 22922941 120 0.000000 589342.013245 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.150932] ll_ost01_027 68580 79503918.627120 18579903 120 0.000000 512100.526367 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.162268] ll_ost01_029 68584 79504427.134616 15391680 120 0.000000 402803.243733 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.173604] ll_ost01_031 68592 79504294.071922 18196240 120 0.000000 448878.743896 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.184941] ll_ost01_032 68601 77191229.142274 26615921 120 0.000000 699619.261641 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.196277] ll_ost01_038 68617 79504430.353943 23494103 120 0.000000 653351.733950 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.207614] ll_ost01_040 68626 79503880.096792 20522381 120 0.000000 503344.185284 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.218958] ll_ost01_061 68713 79504443.740611 20837737 120 0.000000 518708.732443 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.230299] ll_ost_io01_023 68746 79503748.847898 108150195 120 0.000000 8028563.241081 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.241644] ll_ost01_073 68796 77832737.628814 20377990 120 0.000000 514159.200889 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.252994] ll_ost_create01 68913 78687306.614180 2764445 120 0.000000 49079.379587 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.264328] ll_ost_create01 68940 79504442.090806 4154290 120 0.000000 72429.427403 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.275675] ll_ost_io01_075 73706 79504221.456453 56026425 120 0.000000 4580043.205969 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.287010] ll_ost_io01_086 73717 79504174.589751 81224944 120 0.000000 5452307.787313 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.298347] ldlm_cb01_005 76922 228692.973636 25 120 0.000000 0.517162 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.309680] ldlm_cn01_009 81389 79503402.190935 8120243 120 0.000000 187198.577536 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.321017] ll_ost01_082 97756 79503936.722268 16741524 120 0.000000 504809.177940 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.332364] ldlm_bl_14 100095 79504442.078043 6114136 120 0.000000 84113.134599 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.343793] ldlm_bl_57 100138 79504442.068053 6116703 120 0.000000 84118.331033 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.355227] ldlm_cn01_017 111305 79504431.116837 12606326 120 0.000000 265581.293421 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.366653] ldlm_cn01_028 111457 79503402.190715 14542589 120 0.000000 318170.721289 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.378119] ll_ost_io01_110 40830 79504291.499949 34600298 120 0.000000 2165994.368737 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.389465] ll_ost01_091 101301 79504454.059567 17499432 120 0.000000 513817.505611 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.400917] crond 24350 66694403.053392 6 120 0.000000 3.670193 0.000000 1 / [Tue Mar 17 18:02:45 2020][8592650.412260] kworker/41:1 128506 79264683.562186 806 120 0.000000 2.760721 0.000000 1 / [Tue Mar 17 18:02:46 2020][8592650.423684] kworker/41:2 4524 79504438.690863 26987 120 0.000000 389.065343 0.000000 1 / [Tue Mar 17 18:02:46 2020][8592650.435053] [Tue Mar 17 18:02:46 2020][8592650.436728] cpu#42, 1996.233 MHz [Tue Mar 17 18:02:46 2020][8592650.440134] .nr_running : 0 [Tue Mar 17 18:02:46 2020][8592650.444929] .load : 0 [Tue Mar 17 18:02:46 2020][8592650.449720] .nr_switches : 14315035099 [Tue Mar 17 18:02:46 2020][8592650.455397] .nr_load_updates : 3108965234 [Tue Mar 17 18:02:46 2020][8592650.460971] .nr_uninterruptible : -17163320 [Tue Mar 17 18:02:46 2020][8592650.466455] .next_balance : 12887.294968 [Tue Mar 17 18:02:46 2020][8592650.472217] .curr->pid : 0 [Tue Mar 17 18:02:46 2020][8592650.477014] .clock : 8592650476.437117 [Tue Mar 17 18:02:46 2020][8592650.483200] .cpu_load[0] : 0 [Tue Mar 17 18:02:46 2020][8592650.488008] .cpu_load[1] : 0 [Tue Mar 17 18:02:46 2020][8592650.492805] .cpu_load[2] : 0 [Tue Mar 17 18:02:46 2020][8592650.497605] .cpu_load[3] : 0 [Tue Mar 17 18:02:46 2020][8592650.502397] .cpu_load[4] : 0 [Tue Mar 17 18:02:46 2020][8592650.507205] .avg_idle : 8852971 [Tue Mar 17 18:02:46 2020][8592650.512519] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:46 2020][8592650.517832] [Tue Mar 17 18:02:46 2020][8592650.519501] cfs_rq[42]:/ [Tue Mar 17 18:02:46 2020][8592650.522216] .exec_clock : 0.000000 [Tue Mar 17 18:02:46 2020][8592650.527613] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:46 2020][8592650.533013] .min_vruntime : 140330274.698686 [Tue Mar 17 18:02:46 2020][8592650.539106] .max_vruntime : 0.000001 [Tue Mar 17 18:02:46 2020][8592650.544505] .spread : 0.000000 [Tue Mar 17 18:02:46 2020][8592650.549906] .spread0 : -176884868.634065 [Tue Mar 17 18:02:46 2020][8592650.556083] .nr_spread_over : 0 [Tue Mar 17 18:02:46 2020][8592650.560875] .nr_running : 0 [Tue Mar 17 18:02:46 2020][8592650.565668] .load : 0 [Tue Mar 17 18:02:46 2020][8592650.570462] .runnable_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592650.575254] .blocked_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592650.580047] .tg_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592650.584840] .tg_load_contrib : 0 [Tue Mar 17 18:02:46 2020][8592650.589632] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:46 2020][8592650.594426] .tg->runnable_avg : 0 [Tue Mar 17 18:02:46 2020][8592650.599218] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:46 2020][8592650.604012] .throttled : 0 [Tue Mar 17 18:02:46 2020][8592650.608804] .throttle_count : 0 [Tue Mar 17 18:02:46 2020][8592650.613597] .avg->runnable_avg_sum : 12 [Tue Mar 17 18:02:46 2020][8592650.618477] .avg->runnable_avg_period : 46889 [Tue Mar 17 18:02:46 2020][8592650.623618] [Tue Mar 17 18:02:46 2020][8592650.625289] rt_rq[42]:/ [Tue Mar 17 18:02:46 2020][8592650.627912] .rt_nr_running : 0 [Tue Mar 17 18:02:46 2020][8592650.632705] .rt_throttled : 0 [Tue Mar 17 18:02:46 2020][8592650.637500] .rt_time : 0.000000 [Tue Mar 17 18:02:46 2020][8592650.642899] .rt_runtime : 950.000000 [Tue Mar 17 18:02:46 2020][8592650.648471] [Tue Mar 17 18:02:46 2020][8592650.650144] runnable tasks: [Tue Mar 17 18:02:46 2020][8592650.653116] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:46 2020][8592650.664062] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:46 2020][8592650.674959] watchdog/42 221 -11.985993 2148159 0 0.000000 21387.457257 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.686294] migration/42 222 0.000000 82988 0 0.000000 198499.475417 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.697630] ksoftirqd/42 223 140330131.940194 603338672 120 0.000000 6146575.812525 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.708981] kworker/42:1H 5587 68555949.935043 521507055 100 0.000000 2445490.518309 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.720354] kdmwork-253:12 12606 599209.911089 225984 120 0.000000 3134.819212 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.731767] ptlrpcd_02_01 64830 140329766.844737 2548640 120 0.000000 98847.372061 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.743125] ll_ost02_003 67233 140330147.576406 40023764 120 0.000000 1132169.577741 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.754469] ll_ost_io02_005 68357 140329778.135635 134314912 120 0.000000 8666279.497595 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.765822] ll_ost02_023 68553 140330269.441855 46414038 120 0.000000 1357125.272153 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.777156] ll_ost02_026 68559 140330264.260620 35984326 120 0.000000 1015717.206650 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.788500] ll_ost02_042 68634 140330145.160280 39254680 120 0.000000 1127125.906819 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.799841] ll_ost02_049 68659 140330144.329874 50255839 120 0.000000 1431766.539926 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.811176] ll_ost02_053 68684 140330266.359994 40799485 120 0.000000 1142946.341289 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.822515] ll_ost02_061 68737 140330263.433468 43639015 120 0.000000 1241770.660799 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.833857] ll_ost02_066 68758 140330188.902027 42321945 120 0.000000 1237562.710377 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.845188] ll_ost_io02_026 68759 140329820.498286 122909321 120 0.000000 8275351.199022 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.856531] ll_ost_io02_034 68803 140329819.857259 145212374 120 0.000000 9722749.050923 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.867869] ll_ost02_077 68804 140330147.157582 37383372 120 0.000000 1037585.836379 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.879211] ll_ost_io02_044 68855 140329911.052011 96673922 120 0.000000 6234093.051564 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.890554] ldlm_cn02_007 72066 140330262.722499 23048055 120 0.000000 549459.484454 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.901903] ll_ost_io02_058 98929 140330267.179104 142717508 120 0.000000 8942813.706949 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.913267] ldlm_cn02_017 31069 140329324.543040 21798632 120 0.000000 519294.712766 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.924614] ll_ost_io02_064 87189 140329820.117298 65814467 120 0.000000 4335899.549043 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.935953] ll_ost_io02_076 90848 140330151.833839 35968324 120 0.000000 2495538.412470 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.947306] ll_ost_io02_092 40778 140329894.929614 58831055 120 0.000000 3409729.361224 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.958646] ll_ost_io02_099 40788 140330102.877591 126885010 120 0.000000 8504707.182693 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.969999] ll_ost02_099 101324 140330268.364902 31817917 120 0.000000 940522.158212 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.981421] ll_ost02_100 101327 140330175.030584 30145361 120 0.000000 955938.070135 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592650.992861] kworker/42:0H 122772 140330229.099364 863275010 100 0.000000 4547452.391959 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592651.004299] kworker/42:2 4658 140330262.866576 69225 120 0.000000 930.794172 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592651.015645] kworker/42:1 12971 140292449.289817 2 120 0.000000 0.009107 0.000000 2 / [Tue Mar 17 18:02:46 2020][8592651.027025] [Tue Mar 17 18:02:46 2020][8592651.028701] cpu#43, 1996.233 MHz [Tue Mar 17 18:02:46 2020][8592651.032109] .nr_running : 0 [Tue Mar 17 18:02:46 2020][8592651.036901] .load : 0 [Tue Mar 17 18:02:46 2020][8592651.041693] .nr_switches : 10545806342 [Tue Mar 17 18:02:46 2020][8592651.047368] .nr_load_updates : 2658172538 [Tue Mar 17 18:02:46 2020][8592651.052942] .nr_uninterruptible : 216862 [Tue Mar 17 18:02:46 2020][8592651.058169] .next_balance : 12887.295560 [Tue Mar 17 18:02:46 2020][8592651.063929] .curr->pid : 0 [Tue Mar 17 18:02:46 2020][8592651.068726] .clock : 8592651068.439253 [Tue Mar 17 18:02:46 2020][8592651.074904] .cpu_load[0] : 0 [Tue Mar 17 18:02:46 2020][8592651.079713] .cpu_load[1] : 0 [Tue Mar 17 18:02:46 2020][8592651.084507] .cpu_load[2] : 0 [Tue Mar 17 18:02:46 2020][8592651.089301] .cpu_load[3] : 0 [Tue Mar 17 18:02:46 2020][8592651.094102] .cpu_load[4] : 0 [Tue Mar 17 18:02:46 2020][8592651.098907] .avg_idle : 8256567 [Tue Mar 17 18:02:46 2020][8592651.104225] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:46 2020][8592651.109546] [Tue Mar 17 18:02:46 2020][8592651.111215] cfs_rq[43]:/ [Tue Mar 17 18:02:46 2020][8592651.113927] .exec_clock : 0.000000 [Tue Mar 17 18:02:46 2020][8592651.119326] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:46 2020][8592651.124726] .min_vruntime : 85865507.434106 [Tue Mar 17 18:02:46 2020][8592651.130732] .max_vruntime : 0.000001 [Tue Mar 17 18:02:46 2020][8592651.136131] .spread : 0.000000 [Tue Mar 17 18:02:46 2020][8592651.141530] .spread0 : -231349635.898645 [Tue Mar 17 18:02:46 2020][8592651.147709] .nr_spread_over : 0 [Tue Mar 17 18:02:46 2020][8592651.152501] .nr_running : 0 [Tue Mar 17 18:02:46 2020][8592651.157294] .load : 0 [Tue Mar 17 18:02:46 2020][8592651.162087] .runnable_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592651.166880] .blocked_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592651.171673] .tg_load_avg : 0 [Tue Mar 17 18:02:46 2020][8592651.176465] .tg_load_contrib : 0 [Tue Mar 17 18:02:46 2020][8592651.181260] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:46 2020][8592651.186051] .tg->runnable_avg : 0 [Tue Mar 17 18:02:46 2020][8592651.190844] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:46 2020][8592651.195636] .throttled : 0 [Tue Mar 17 18:02:46 2020][8592651.200428] .throttle_count : 0 [Tue Mar 17 18:02:46 2020][8592651.205221] .avg->runnable_avg_sum : 12 [Tue Mar 17 18:02:46 2020][8592651.210101] .avg->runnable_avg_period : 48121 [Tue Mar 17 18:02:46 2020][8592651.215244] [Tue Mar 17 18:02:46 2020][8592651.216913] rt_rq[43]:/ [Tue Mar 17 18:02:46 2020][8592651.219540] .rt_nr_running : 0 [Tue Mar 17 18:02:46 2020][8592651.224331] .rt_throttled : 0 [Tue Mar 17 18:02:46 2020][8592651.229125] .rt_time : 0.000000 [Tue Mar 17 18:02:46 2020][8592651.234523] .rt_runtime : 950.000000 [Tue Mar 17 18:02:46 2020][8592651.240096] [Tue Mar 17 18:02:46 2020][8592651.241768] runnable tasks: [Tue Mar 17 18:02:46 2020][8592651.244741] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:46 2020][8592651.255689] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:46 2020][8592651.266582] watchdog/43 226 -11.986785 2148159 0 0.000000 14994.403627 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.277918] migration/43 227 0.000000 31032 0 0.000000 24018.185162 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.289253] ksoftirqd/43 228 85864988.096926 98838106 120 0.000000 1098928.127679 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.300591] kworker/43:0H 230 1125.671940 10 100 0.000000 0.032966 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.311934] khugepaged 281 85865492.339590 1967205 139 0.000000 1479820.300464 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.323274] ata_sff 1557 100.346766 2 100 0.000000 0.012775 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.334613] kworker/43:1H 5584 85864995.418890 160342825 100 0.000000 707714.571927 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.345985] kdmwork-253:7 12523 85865000.823063 867565927 120 0.000000 31888282.358066 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.357327] kdmwork-253:15 12660 234168.816594 94930 120 0.000000 1011.163816 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.368697] bioset 18429 11360.919016 2 100 0.000000 0.007815 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.380044] kdmflush 24526 14471.997946 2 100 0.000000 0.009539 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.391386] gmain 50202 30934.283146 3 120 0.000000 0.064512 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.402732] kiblnd_sd_03_00 64794 85865507.548119 9107765023 120 0.000000 69267553.740736 0.000000 3 / [Tue Mar 17 18:02:46 2020][8592651.414159] ptlrpcd_03_08 64849 85864472.290275 3444896 120 0.000000 119719.228077 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.425499] ldiskfs-rsv-con 66709 34125.594488 2 100 0.000000 0.012073 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.436887] ll_ost03_055 68677 84698373.649598 17463362 120 0.000000 524282.151116 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.448235] ldlm_cn03_004 68834 85865359.745160 14373554 120 0.000000 345540.360096 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.459587] ll_ost_create03 69530 84935354.268605 3545867 120 0.000000 60622.568744 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.470991] ll_ost_io03_088 40814 85864739.161430 161835169 120 0.000000 9791215.996939 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.482330] ll_ost_io03_097 40841 85864528.817171 38367184 120 0.000000 2194880.445081 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.493683] ll_ost03_070 70316 85865072.329856 15336038 120 0.000000 421918.221367 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.505034] ll_ost03_103 113486 85460809.759102 5513833 120 0.000000 159550.302086 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.516463] kworker/43:1 69254 85865480.354960 288502 120 0.000000 4865.368438 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.527802] kworker/43:2 12370 84492304.967958 2 120 0.000000 0.009799 0.000000 3 / [Tue Mar 17 18:02:47 2020][8592651.539177] [Tue Mar 17 18:02:47 2020][8592651.540853] cpu#44, 1996.233 MHz [Tue Mar 17 18:02:47 2020][8592651.544261] .nr_running : 0 [Tue Mar 17 18:02:47 2020][8592651.549053] .load : 0 [Tue Mar 17 18:02:47 2020][8592651.553846] .nr_switches : 8991099586 [Tue Mar 17 18:02:47 2020][8592651.559434] .nr_load_updates : 2318171477 [Tue Mar 17 18:02:47 2020][8592651.565030] .nr_uninterruptible : 439480 [Tue Mar 17 18:02:47 2020][8592651.570261] .next_balance : 12887.296070 [Tue Mar 17 18:02:47 2020][8592651.576005] .curr->pid : 0 [Tue Mar 17 18:02:47 2020][8592651.580816] .clock : 8592651580.436439 [Tue Mar 17 18:02:47 2020][8592651.586996] .cpu_load[0] : 0 [Tue Mar 17 18:02:47 2020][8592651.591789] .cpu_load[1] : 0 [Tue Mar 17 18:02:47 2020][8592651.596594] .cpu_load[2] : 0 [Tue Mar 17 18:02:47 2020][8592651.601390] .cpu_load[3] : 0 [Tue Mar 17 18:02:47 2020][8592651.606185] .cpu_load[4] : 0 [Tue Mar 17 18:02:47 2020][8592651.610978] .avg_idle : 3848267 [Tue Mar 17 18:02:47 2020][8592651.616325] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:47 2020][8592651.621636] [Tue Mar 17 18:02:47 2020][8592651.623305] cfs_rq[44]:/ [Tue Mar 17 18:02:47 2020][8592651.626019] .exec_clock : 0.000000 [Tue Mar 17 18:02:47 2020][8592651.631418] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:47 2020][8592651.636818] .min_vruntime : 152947722.265105 [Tue Mar 17 18:02:47 2020][8592651.642908] .max_vruntime : 0.000001 [Tue Mar 17 18:02:47 2020][8592651.648307] .spread : 0.000000 [Tue Mar 17 18:02:47 2020][8592651.653707] .spread0 : -164267421.067646 [Tue Mar 17 18:02:47 2020][8592651.659888] .nr_spread_over : 0 [Tue Mar 17 18:02:47 2020][8592651.664679] .nr_running : 0 [Tue Mar 17 18:02:47 2020][8592651.669471] .load : 0 [Tue Mar 17 18:02:47 2020][8592651.674265] .runnable_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592651.679058] .blocked_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592651.683852] .tg_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592651.688643] .tg_load_contrib : 0 [Tue Mar 17 18:02:47 2020][8592651.693437] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:47 2020][8592651.698230] .tg->runnable_avg : 0 [Tue Mar 17 18:02:47 2020][8592651.703021] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:47 2020][8592651.707815] .throttled : 0 [Tue Mar 17 18:02:47 2020][8592651.712608] .throttle_count : 0 [Tue Mar 17 18:02:47 2020][8592651.717400] .avg->runnable_avg_sum : 71 [Tue Mar 17 18:02:47 2020][8592651.722278] .avg->runnable_avg_period : 47068 [Tue Mar 17 18:02:47 2020][8592651.727421] [Tue Mar 17 18:02:47 2020][8592651.729091] rt_rq[44]:/ [Tue Mar 17 18:02:47 2020][8592651.731717] .rt_nr_running : 0 [Tue Mar 17 18:02:47 2020][8592651.736509] .rt_throttled : 0 [Tue Mar 17 18:02:47 2020][8592651.741302] .rt_time : 0.005921 [Tue Mar 17 18:02:47 2020][8592651.746701] .rt_runtime : 950.000000 [Tue Mar 17 18:02:47 2020][8592651.752275] [Tue Mar 17 18:02:47 2020][8592651.753947] runnable tasks: [Tue Mar 17 18:02:47 2020][8592651.756921] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:47 2020][8592651.767865] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:47 2020][8592651.778761] watchdog/44 231 -11.988228 2148160 0 0.000000 16952.089466 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.790097] migration/44 232 0.000000 68139 0 0.000000 50549.980611 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.801431] ksoftirqd/44 233 152947710.199651 158553053 120 0.000000 1723017.280156 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.812767] kworker/44:0H 235 5974.774735 10 100 0.000000 0.033313 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.824158] kdmwork-253:24 12691 178882.054778 96230 120 0.000000 1024.693279 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.835504] kworker/44:1H 14170 152947638.291667 243914839 100 0.000000 1048548.512074 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.846840] kdmwork-253:42 17999 152947650.291589 828933780 120 0.000000 27654178.580035 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.858183] kdmwork-253:52 18152 152947646.161221 871937152 120 0.000000 31258958.682872 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.869537] kdmwork-253:94 19046 178881.840762 74572 120 0.000000 929.959177 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.880891] polkitd 50238 27946.579909 3 120 0.000000 0.093937 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.892235] kiblnd_sd_00_01 64783 329700539.452192 11544586357 120 0.000000 74107970.393367 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.903743] kiblnd_sd_00_02 64784 152947722.172379 11543816652 120 0.000000 74158483.392465 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.915257] ptlrpcd_00_04 64809 152947373.235590 4137855 120 0.000000 143256.651250 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.926618] ll_ost00_008 68340 147349900.673916 25338505 120 0.000000 582156.485076 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.937976] ll_ost00_020 68565 147349900.673865 35166341 120 0.000000 743469.405065 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.949326] ll_ost00_041 68658 147349900.686762 44646928 120 0.000000 990798.910680 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.960664] ll_ost00_055 68726 151747230.496249 45873264 120 0.000000 1037644.275963 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.972005] ll_ost00_060 68742 147349900.767192 37413833 120 0.000000 866753.043271 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.983354] ll_ost_io00_050 68898 152947507.854795 66747690 120 0.000000 4962261.461108 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592651.994692] ll_ost_create00 68905 151916564.434804 251839 120 0.000000 3174.029916 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.006056] ldlm_bl_35 100116 152947710.276216 6113184 120 0.000000 146519.023928 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.017482] ldlm_bl_66 100147 152947710.243052 6110499 120 0.000000 83562.467036 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.028901] ldlm_bl_74 100155 152947709.267656 6113640 120 0.000000 84368.344049 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.040355] ll_ost_io00_067 90863 152947504.232661 72930897 120 0.000000 4664112.268013 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.051696] ll_ost_io00_072 10667 152947443.927684 53346726 120 0.000000 3509743.363464 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.063067] ll_ost00_091 74615 150534931.304504 18133014 120 0.000000 400114.682818 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.074404] ll_ost00_104 85273 147349900.700757 16103686 120 0.000000 360993.379832 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.085749] kworker/44:2 29036 150640183.011532 1240 120 0.000000 25.851585 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.097079] kworker/44:0 31770 152947688.910670 199151 120 0.000000 2998.041108 0.000000 0 / [Tue Mar 17 18:02:47 2020][8592652.108478] [Tue Mar 17 18:02:47 2020][8592652.110152] cpu#45, 1996.233 MHz [Tue Mar 17 18:02:47 2020][8592652.113560] .nr_running : 0 [Tue Mar 17 18:02:47 2020][8592652.118352] .load : 0 [Tue Mar 17 18:02:47 2020][8592652.123145] .nr_switches : 7801903572 [Tue Mar 17 18:02:47 2020][8592652.128733] .nr_load_updates : 2180606907 [Tue Mar 17 18:02:47 2020][8592652.134322] .nr_uninterruptible : 206850 [Tue Mar 17 18:02:47 2020][8592652.139551] .next_balance : 12887.296643 [Tue Mar 17 18:02:47 2020][8592652.145311] .curr->pid : 0 [Tue Mar 17 18:02:47 2020][8592652.150108] .clock : 8592652149.441991 [Tue Mar 17 18:02:47 2020][8592652.156286] .cpu_load[0] : 0 [Tue Mar 17 18:02:47 2020][8592652.161094] .cpu_load[1] : 0 [Tue Mar 17 18:02:47 2020][8592652.165888] .cpu_load[2] : 0 [Tue Mar 17 18:02:47 2020][8592652.170683] .cpu_load[3] : 0 [Tue Mar 17 18:02:47 2020][8592652.175489] .cpu_load[4] : 0 [Tue Mar 17 18:02:47 2020][8592652.180285] .avg_idle : 10000000 [Tue Mar 17 18:02:47 2020][8592652.185684] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:47 2020][8592652.191013] [Tue Mar 17 18:02:47 2020][8592652.192685] cfs_rq[45]:/ [Tue Mar 17 18:02:47 2020][8592652.195396] .exec_clock : 0.000000 [Tue Mar 17 18:02:47 2020][8592652.200794] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:47 2020][8592652.206195] .min_vruntime : 78880500.484838 [Tue Mar 17 18:02:47 2020][8592652.212200] .max_vruntime : 0.000001 [Tue Mar 17 18:02:47 2020][8592652.217601] .spread : 0.000000 [Tue Mar 17 18:02:47 2020][8592652.222999] .spread0 : -238334642.847913 [Tue Mar 17 18:02:47 2020][8592652.229178] .nr_spread_over : 0 [Tue Mar 17 18:02:47 2020][8592652.233972] .nr_running : 0 [Tue Mar 17 18:02:47 2020][8592652.238763] .load : 0 [Tue Mar 17 18:02:47 2020][8592652.243557] .runnable_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592652.248349] .blocked_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592652.253142] .tg_load_avg : 0 [Tue Mar 17 18:02:47 2020][8592652.257936] .tg_load_contrib : 0 [Tue Mar 17 18:02:47 2020][8592652.262727] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:47 2020][8592652.267520] .tg->runnable_avg : 0 [Tue Mar 17 18:02:47 2020][8592652.272312] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:47 2020][8592652.277105] .throttled : 0 [Tue Mar 17 18:02:47 2020][8592652.281898] .throttle_count : 0 [Tue Mar 17 18:02:47 2020][8592652.286690] .avg->runnable_avg_sum : 0 [Tue Mar 17 18:02:47 2020][8592652.291485] .avg->runnable_avg_period : 46672 [Tue Mar 17 18:02:47 2020][8592652.296624] [Tue Mar 17 18:02:47 2020][8592652.298294] rt_rq[45]:/ [Tue Mar 17 18:02:47 2020][8592652.300923] .rt_nr_running : 0 [Tue Mar 17 18:02:47 2020][8592652.305715] .rt_throttled : 0 [Tue Mar 17 18:02:47 2020][8592652.310508] .rt_time : 0.000000 [Tue Mar 17 18:02:47 2020][8592652.315907] .rt_runtime : 950.000000 [Tue Mar 17 18:02:47 2020][8592652.321480] [Tue Mar 17 18:02:47 2020][8592652.323152] runnable tasks: [Tue Mar 17 18:02:47 2020][8592652.326123] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:47 2020][8592652.337071] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:47 2020][8592652.347966] watchdog/45 236 -11.985552 2148160 0 0.000000 15072.746834 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.359302] migration/45 237 0.000000 36076 0 0.000000 17994.356687 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.370636] ksoftirqd/45 238 78880272.318557 71864571 120 0.000000 852993.943928 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.381972] kworker/45:0H 240 8866.158435 10 100 0.000000 0.060123 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.393360] kdmwork-253:25 12694 396236.350343 232223 120 0.000000 3178.332193 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.404711] kworker/45:1H 18011 78880272.308983 129230185 100 0.000000 662735.638946 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.416065] bioset 18700 9377.669570 2 100 0.000000 0.010891 0.000000 1 / [Tue Mar 17 18:02:47 2020][8592652.427423] polkitd 50170 78880485.128454 2529924 120 0.000000 107089.384901 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.438778] ptlrpcd_01_10 64827 78880485.742474 4399847 120 0.000000 137457.525610 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.450136] ll_ost01_007 68148 78358916.528270 20770698 120 0.000000 599236.538071 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.461481] ll_ost_io01_007 68368 78880288.973298 62322233 120 0.000000 4139348.149229 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.472834] ll_ost01_016 68537 78879883.174143 18166755 120 0.000000 494562.407174 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.484181] ll_ost01_028 68581 76591059.676436 22242131 120 0.000000 529433.373621 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.495531] ll_ost01_063 68727 76591059.660054 19052948 120 0.000000 518879.167058 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.506886] ll_ost_io01_043 68866 78880250.892547 88922526 120 0.000000 5148106.028337 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.518247] ll_ost01_084 97758 78880486.069559 18701763 120 0.000000 478956.420218 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.529598] ldlm_bl_42 100123 78880487.939230 6104536 120 0.000000 83383.509011 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.541016] ldlm_bl_44 100125 78880487.638411 6105298 120 0.000000 83568.672396 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.552447] ldlm_bl_75 100156 78880488.492592 6111637 120 0.000000 83956.008949 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.563871] ldlm_bl_82 100163 78880488.493184 6114113 120 0.000000 83639.355053 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.575299] ldlm_cn01_021 111313 78879563.636298 11263008 120 0.000000 244983.485020 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.586726] ldlm_cn01_024 111316 78879554.675693 15480837 120 0.000000 319683.120699 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.598181] ll_ost_io01_094 40248 78880499.100793 61399343 120 0.000000 3245808.280383 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.609539] ll_ost01_101 101311 78880483.045438 14442877 120 0.000000 434741.068075 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.620980] kworker/45:2 80081 78880488.495728 2381714 120 0.000000 49965.550077 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.632323] kworker/45:0 2020 78558129.067850 2 120 0.000000 0.009878 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.643669] kworker/u258:4 13546 78880481.799243 767 120 0.000000 49.008404 0.000000 1 / [Tue Mar 17 18:02:48 2020][8592652.655029] [Tue Mar 17 18:02:48 2020][8592652.656703] cpu#46, 1996.233 MHz [Tue Mar 17 18:02:48 2020][8592652.660109] .nr_running : 0 [Tue Mar 17 18:02:48 2020][8592652.664903] .load : 0 [Tue Mar 17 18:02:48 2020][8592652.669696] .nr_switches : 13641640889 [Tue Mar 17 18:02:48 2020][8592652.675376] .nr_load_updates : 3029002039 [Tue Mar 17 18:02:48 2020][8592652.680954] .nr_uninterruptible : -16926045 [Tue Mar 17 18:02:48 2020][8592652.686445] .next_balance : 12887.297187 [Tue Mar 17 18:02:48 2020][8592652.692209] .curr->pid : 0 [Tue Mar 17 18:02:48 2020][8592652.697004] .clock : 8592652696.443058 [Tue Mar 17 18:02:48 2020][8592652.703182] .cpu_load[0] : 0 [Tue Mar 17 18:02:48 2020][8592652.707989] .cpu_load[1] : 0 [Tue Mar 17 18:02:48 2020][8592652.712786] .cpu_load[2] : 0 [Tue Mar 17 18:02:48 2020][8592652.717579] .cpu_load[3] : 0 [Tue Mar 17 18:02:48 2020][8592652.722371] .cpu_load[4] : 0 [Tue Mar 17 18:02:48 2020][8592652.727178] .avg_idle : 4031096 [Tue Mar 17 18:02:48 2020][8592652.732493] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:48 2020][8592652.737807] [Tue Mar 17 18:02:48 2020][8592652.739476] cfs_rq[46]:/ [Tue Mar 17 18:02:48 2020][8592652.742189] .exec_clock : 0.000000 [Tue Mar 17 18:02:48 2020][8592652.747587] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:48 2020][8592652.752987] .min_vruntime : 141826198.675608 [Tue Mar 17 18:02:48 2020][8592652.759081] .max_vruntime : 0.000001 [Tue Mar 17 18:02:48 2020][8592652.764480] .spread : 0.000000 [Tue Mar 17 18:02:48 2020][8592652.769879] .spread0 : -175388944.692879 [Tue Mar 17 18:02:48 2020][8592652.776058] .nr_spread_over : 0 [Tue Mar 17 18:02:48 2020][8592652.780851] .nr_running : 0 [Tue Mar 17 18:02:48 2020][8592652.785645] .load : 0 [Tue Mar 17 18:02:48 2020][8592652.790435] .runnable_load_avg : 0 [Tue Mar 17 18:02:48 2020][8592652.795228] .blocked_load_avg : 0 [Tue Mar 17 18:02:48 2020][8592652.800023] .tg_load_avg : 0 [Tue Mar 17 18:02:48 2020][8592652.804814] .tg_load_contrib : 0 [Tue Mar 17 18:02:48 2020][8592652.809607] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:48 2020][8592652.814400] .tg->runnable_avg : 0 [Tue Mar 17 18:02:48 2020][8592652.819191] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:48 2020][8592652.823984] .throttled : 0 [Tue Mar 17 18:02:48 2020][8592652.828777] .throttle_count : 0 [Tue Mar 17 18:02:48 2020][8592652.833570] .avg->runnable_avg_sum : 1 [Tue Mar 17 18:02:48 2020][8592652.838364] .avg->runnable_avg_period : 47653 [Tue Mar 17 18:02:48 2020][8592652.843505] [Tue Mar 17 18:02:48 2020][8592652.845175] rt_rq[46]:/ [Tue Mar 17 18:02:48 2020][8592652.847802] .rt_nr_running : 0 [Tue Mar 17 18:02:48 2020][8592652.852593] .rt_throttled : 0 [Tue Mar 17 18:02:48 2020][8592652.857386] .rt_time : 0.000000 [Tue Mar 17 18:02:48 2020][8592652.862787] .rt_runtime : 950.000000 [Tue Mar 17 18:02:48 2020][8592652.868359] [Tue Mar 17 18:02:48 2020][8592652.870031] runnable tasks: [Tue Mar 17 18:02:48 2020][8592652.873004] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:48 2020][8592652.883951] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:48 2020][8592652.894846] watchdog/46 241 -11.987046 2148162 0 0.000000 21954.252047 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.906180] migration/46 242 0.000000 82966 0 0.000000 197358.941505 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.917516] ksoftirqd/46 243 141826185.720582 618330440 120 0.000000 6203693.711232 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.928852] kworker/46:0H 245 97983497.655466 19 100 0.000000 0.226529 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.940205] ccp-1-q4 6420 1801.538195 2 120 0.000000 0.012935 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.951543] ccp-3-q3 6477 1830.642085 2 120 0.000000 0.012814 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.962879] ccp-3-q4 6478 1842.649223 2 120 0.000000 0.007144 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.974251] kdmwork-253:22 12685 141825040.433920 823110142 120 0.000000 29947642.465295 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.985617] kdmwork-253:73 18413 141825041.709013 808247945 120 0.000000 27754077.918617 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592652.996977] kdmwork-253:111 25247 577709.620487 84166 120 0.000000 1031.263799 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.008320] systemd-logind 50269 141826067.187448 4584305 120 0.000000 239395.235506 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.019654] writer#0 50894 141826186.665299 49030446 120 0.000000 722814.997514 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.031008] ptlrpcd_02_09 64838 141824747.818410 2543569 120 0.000000 98745.287622 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.042345] bioset 65597 30912.645835 2 100 0.000000 0.012124 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.053681] bioset 66007 31274.672839 2 100 0.000000 0.030557 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.065016] bioset 66010 31286.682341 2 100 0.000000 0.010440 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.076352] bioset 66113 31364.257627 2 100 0.000000 0.011351 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.087687] bioset 66114 31376.266422 2 100 0.000000 0.009347 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.099030] ldlm_cb02_001 66880 141251738.268352 2034 120 0.000000 36.496015 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.110377] ldiskfs-rsv-con 67058 32549.318373 2 100 0.000000 0.011842 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.121712] ldiskfs-rsv-con 67082 32567.759170 2 100 0.000000 0.012594 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.133057] ll_ost02_008 68322 139081144.171052 28275391 120 0.000000 778638.471337 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.144398] ll_ost_io02_004 68354 141824976.012946 81640807 120 0.000000 6119988.998729 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.155735] ll_ost_io02_012 68393 141824796.603025 103662828 120 0.000000 7250098.035376 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.167071] ll_ost_io02_015 68491 141824773.018694 66384538 120 0.000000 4734472.022329 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.178404] ll_ost02_021 68500 141825027.259757 30531706 120 0.000000 868022.821177 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.189748] ll_ost02_024 68555 141824964.951953 44066936 120 0.000000 1271701.320344 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.201087] ll_ost02_030 68579 141825107.365609 29042765 120 0.000000 791745.369846 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.212426] ll_ost02_046 68650 141825029.491105 45939512 120 0.000000 1304893.522740 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.223764] ll_ost02_048 68656 141824757.101565 36242873 120 0.000000 1022812.397901 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.235104] ll_ost02_055 68695 141826068.003832 27205464 120 0.000000 825755.431590 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.246439] ll_ost02_059 68725 141824745.882354 40273307 120 0.000000 1191248.889952 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.257775] ll_ost_io02_025 68754 141824950.272854 121411860 120 0.000000 8078486.011495 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.269115] ll_ost_io02_033 68802 141824753.159269 95666476 120 0.000000 6113065.479446 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.280455] ll_ost_io02_038 68837 141824963.139046 126021653 120 0.000000 8014483.043934 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.291792] ll_ost_io02_045 68859 141824752.778957 107026754 120 0.000000 7021592.623447 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.303133] ldlm_cn02_005 69306 141824544.312781 27756619 120 0.000000 657769.578915 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.314489] ll_ost02_091 98682 139081144.170990 38470478 120 0.000000 1092826.964036 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.325825] ll_ost_io02_060 98931 141824958.707478 52740068 120 0.000000 3646374.559640 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.337176] grep 102170 141826198.675608 86606 120 0.000000 838.116872 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.348606] ldlm_cn02_015 111323 141770104.157429 21048629 120 0.000000 483830.083649 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.360036] ldlm_cn02_016 31068 141770104.157369 18314204 120 0.000000 450883.976649 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.371371] ldlm_cn02_027 64120 141770104.157378 23786285 120 0.000000 554774.936235 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.382727] ldlm_cb02_006 40400 141251741.728506 1890 120 0.000000 38.481397 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.394067] ll_ost_io02_094 40780 141824747.554350 123101885 120 0.000000 8587594.374244 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.405410] ll_ost_io02_108 40827 141824793.967216 104095225 120 0.000000 6640584.663335 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.416764] crond 57399 36973621.186838 6 120 0.000000 3.499181 0.000000 2 / [Tue Mar 17 18:02:48 2020][8592653.428115] kworker/46:1H 51157 141825064.308645 561059514 100 0.000000 2862876.157881 0.000000 2 / [Tue Mar 17 18:02:49 2020][8592653.439465] kworker/46:1 9243 141826159.831976 26682 120 0.000000 506.114260 0.000000 2 / [Tue Mar 17 18:02:49 2020][8592653.450799] sshd 11736 141750811.617293 104 120 0.000000 14.435717 0.000000 2 / [Tue Mar 17 18:02:49 2020][8592653.462138] kworker/46:2 13418 141804312.674976 2 120 0.000000 0.010290 0.000000 2 / [Tue Mar 17 18:02:49 2020][8592653.473519] [Tue Mar 17 18:02:49 2020][8592653.475196] cpu#47, 1996.233 MHz [Tue Mar 17 18:02:49 2020][8592653.478604] .nr_running : 0 [Tue Mar 17 18:02:49 2020][8592653.483397] .load : 0 [Tue Mar 17 18:02:49 2020][8592653.488189] .nr_switches : 9878802376 [Tue Mar 17 18:02:49 2020][8592653.493777] .nr_load_updates : 2584761065 [Tue Mar 17 18:02:49 2020][8592653.499352] .nr_uninterruptible : 225313 [Tue Mar 17 18:02:49 2020][8592653.504578] .next_balance : 12887.298008 [Tue Mar 17 18:02:49 2020][8592653.510338] .curr->pid : 0 [Tue Mar 17 18:02:49 2020][8592653.515136] .clock : 8592653514.443361 [Tue Mar 17 18:02:49 2020][8592653.521313] .cpu_load[0] : 0 [Tue Mar 17 18:02:49 2020][8592653.526121] .cpu_load[1] : 0 [Tue Mar 17 18:02:49 2020][8592653.530915] .cpu_load[2] : 0 [Tue Mar 17 18:02:49 2020][8592653.535710] .cpu_load[3] : 0 [Tue Mar 17 18:02:49 2020][8592653.540514] .cpu_load[4] : 0 [Tue Mar 17 18:02:49 2020][8592653.545313] .avg_idle : 10000000 [Tue Mar 17 18:02:49 2020][8592653.550711] .max_idle_balance_cost : 5000000 [Tue Mar 17 18:02:49 2020][8592653.556037] [Tue Mar 17 18:02:49 2020][8592653.557711] cfs_rq[47]:/ [Tue Mar 17 18:02:49 2020][8592653.560422] .exec_clock : 0.000000 [Tue Mar 17 18:02:49 2020][8592653.565822] .MIN_vruntime : 0.000001 [Tue Mar 17 18:02:49 2020][8592653.571222] .min_vruntime : 83923538.662633 [Tue Mar 17 18:02:49 2020][8592653.577229] .max_vruntime : 0.000001 [Tue Mar 17 18:02:49 2020][8592653.582626] .spread : 0.000000 [Tue Mar 17 18:02:49 2020][8592653.588026] .spread0 : -233291604.941158 [Tue Mar 17 18:02:49 2020][8592653.594205] .nr_spread_over : 0 [Tue Mar 17 18:02:49 2020][8592653.598999] .nr_running : 0 [Tue Mar 17 18:02:49 2020][8592653.603791] .load : 0 [Tue Mar 17 18:02:49 2020][8592653.608585] .runnable_load_avg : 0 [Tue Mar 17 18:02:49 2020][8592653.613377] .blocked_load_avg : 0 [Tue Mar 17 18:02:49 2020][8592653.618168] .tg_load_avg : 0 [Tue Mar 17 18:02:49 2020][8592653.622961] .tg_load_contrib : 0 [Tue Mar 17 18:02:49 2020][8592653.627756] .tg_runnable_contrib : 0 [Tue Mar 17 18:02:49 2020][8592653.632547] .tg->runnable_avg : 0 [Tue Mar 17 18:02:49 2020][8592653.637339] .tg->cfs_bandwidth.timer_active: 0 [Tue Mar 17 18:02:49 2020][8592653.642134] .throttled : 0 [Tue Mar 17 18:02:49 2020][8592653.646925] .throttle_count : 0 [Tue Mar 17 18:02:49 2020][8592653.651718] .avg->runnable_avg_sum : 28 [Tue Mar 17 18:02:49 2020][8592653.656598] .avg->runnable_avg_period : 47248 [Tue Mar 17 18:02:49 2020][8592653.661742] [Tue Mar 17 18:02:49 2020][8592653.663411] rt_rq[47]:/ [Tue Mar 17 18:02:49 2020][8592653.666035] .rt_nr_running : 0 [Tue Mar 17 18:02:49 2020][8592653.670829] .rt_throttled : 0 [Tue Mar 17 18:02:49 2020][8592653.675621] .rt_time : 0.000000 [Tue Mar 17 18:02:49 2020][8592653.681021] .rt_runtime : 950.000000 [Tue Mar 17 18:02:49 2020][8592653.686592] [Tue Mar 17 18:02:49 2020][8592653.688265] runnable tasks: [Tue Mar 17 18:02:49 2020][8592653.691237] task PID tree-key switches prio wait-time sum-exec sum-sleep [Tue Mar 17 18:02:49 2020][8592653.702185] ---------------------------------------------------------------------------------------------------------- [Tue Mar 17 18:02:49 2020][8592653.713082] watchdog/47 246 -11.986555 2148160 0 0.000000 14367.244582 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.724413] migration/47 247 0.000000 39008 0 0.000000 30573.099038 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.735749] ksoftirqd/47 248 83923430.394743 100708571 120 0.000000 1111578.857722 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.747087] kworker/47:0H 250 1691.693602 10 100 0.000000 0.040756 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.758435] kworker/47:1H 5585 83923430.313712 162520015 100 0.000000 708520.995477 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.769778] kmpathd 11172 4482.818524 2 100 0.000000 0.010700 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.781113] kmpath_handlerd 11173 4494.826939 2 100 0.000000 0.008947 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.792449] multipathd 11204 0.000000 2691 0 0.000000 295.298007 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.803786] multipathd 11207 0.000000 1718517 0 0.000000 38515.201347 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.815129] multipathd 17982 0.000000 4 0 0.000000 1.045248 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.826465] multipathd 18000 0.000000 3 0 0.000000 1.406439 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.837801] multipathd 18004 0.000000 4 0 0.000000 0.846515 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.849139] multipathd 18010 0.000000 4 0 0.000000 1.330516 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.860474] multipathd 18015 0.000000 4 0 0.000000 0.873416 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.871809] multipathd 18021 0.000000 4 0 0.000000 1.402992 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.883146] multipathd 18031 0.000000 4 0 0.000000 1.052653 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.894482] multipathd 18036 0.000000 3 0 0.000000 1.196473 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.905817] multipathd 18064 0.000000 4 0 0.000000 1.122223 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.917154] multipathd 18069 0.000000 3 0 0.000000 1.276584 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.928490] multipathd 18136 0.000000 4 0 0.000000 1.121041 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.939825] multipathd 18153 0.000000 3 0 0.000000 1.393073 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.951163] multipathd 18171 0.000000 4 0 0.000000 0.906679 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.962499] multipathd 18182 0.000000 3 0 0.000000 1.268058 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.973834] multipathd 18194 0.000000 4 0 0.000000 0.856524 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.985171] multipathd 18224 0.000000 4 0 0.000000 1.263010 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592653.996507] multipathd 18234 0.000000 4 0 0.000000 0.821888 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.007842] multipathd 18245 0.000000 3 0 0.000000 1.214859 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.019179] multipathd 18269 0.000000 4 0 0.000000 0.805148 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.030515] multipathd 18278 0.000000 4 0 0.000000 1.590617 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.041851] multipathd 18286 0.000000 4 0 0.000000 0.845553 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.053186] multipathd 18294 0.000000 4 0 0.000000 1.223193 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.064522] multipathd 18298 0.000000 4 0 0.000000 0.753138 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.075858] multipathd 18302 0.000000 3 0 0.000000 1.060688 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.087195] multipathd 18321 0.000000 4 0 0.000000 1.057661 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.098531] multipathd 18325 0.000000 3 0 0.000000 1.226449 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.109868] multipathd 18331 0.000000 4 0 0.000000 0.864698 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.121203] multipathd 18337 0.000000 2 0 0.000000 1.257258 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.132539] multipathd 18344 0.000000 4 0 0.000000 0.844020 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.143875] multipathd 18385 0.000000 3 0 0.000000 1.786044 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.155211] multipathd 18391 0.000000 3 0 0.000000 1.065787 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.166547] multipathd 18404 0.000000 3 0 0.000000 1.649497 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.177884] multipathd 18414 0.000000 3 0 0.000000 1.153092 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.189219] multipathd 18423 0.000000 2 0 0.000000 1.453027 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.200556] multipathd 18434 0.000000 4 0 0.000000 1.121662 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.211892] multipathd 18443 0.000000 2 0 0.000000 1.340364 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.223229] multipathd 18490 0.000000 4 0 0.000000 1.525754 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.234566] multipathd 18495 0.000000 3 0 0.000000 1.510485 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.245900] multipathd 18508 0.000000 4 0 0.000000 1.049146 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.257237] multipathd 18519 0.000000 3 0 0.000000 1.369228 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.268573] multipathd 18526 0.000000 4 0 0.000000 0.931565 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.279908] multipathd 18536 0.000000 3 0 0.000000 1.455592 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.291245] multipathd 18544 0.000000 4 0 0.000000 0.951783 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.302582] multipathd 18556 0.000000 3 0 0.000000 1.317071 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.313916] multipathd 18563 0.000000 4 0 0.000000 1.097198 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.325253] multipathd 18575 0.000000 3 0 0.000000 1.362286 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.336589] multipathd 18587 0.000000 4 0 0.000000 0.978233 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.347924] multipathd 18597 0.000000 3 0 0.000000 1.437198 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.359261] multipathd 18667 0.000000 3 0 0.000000 1.332570 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.370597] multipathd 18683 0.000000 4 0 0.000000 1.613981 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.381933] multipathd 18696 0.000000 4 0 0.000000 1.089493 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.393270] multipathd 18702 0.000000 3 0 0.000000 1.501869 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.404606] multipathd 18816 0.000000 1 0 0.000000 1.838723 0.000000 3 / [Tue Mar 17 18:02:49 2020][8592654.415942] multipathd 19050 0.000000 3 0 0.000000 2.075398 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.427278] multipathd 19078 0.000000 4 0 0.000000 1.214517 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.438614] multipathd 19148 0.000000 3 0 0.000000 1.638276 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.449950] multipathd 19162 0.000000 4 0 0.000000 1.017427 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.461285] multipathd 19263 0.000000 2 0 0.000000 1.759272 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.472622] multipathd 19377 0.000000 3 0 0.000000 1.452847 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.483959] multipathd 19399 0.000000 2 0 0.000000 1.606566 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.495294] multipathd 19615 0.000000 4 0 0.000000 1.896021 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.506631] multipathd 19665 0.000000 3 0 0.000000 1.677068 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.517967] multipathd 19995 0.000000 4 0 0.000000 1.808755 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.529302] multipathd 20040 0.000000 3 0 0.000000 1.931588 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.540638] multipathd 20098 0.000000 4 0 0.000000 1.650017 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.551975] multipathd 20673 0.000000 2 0 0.000000 2.516339 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.563310] multipathd 21032 0.000000 3 0 0.000000 1.813888 0.000000[8592654.572463] LustreError: dumping log to /tmp/lustre-log.1584493370.68356 [Tue Mar 17 18:02:50 2020] 3 / [Tue Mar 17 18:02:50 2020][8592654.581519] multipathd 24529 0.000000 1 0 0.000000 2.370675 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.592856] multipathd 25217 0.000000 1 0 0.000000 2.072784 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.604193] multipathd 25235 0.000000 1 0 0.000000 1.724509 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.615527] multipathd 25251 0.000000 1 0 0.000000 1.477943 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.626864] multipathd 25383 0.000000 1 0 0.000000 1.964279 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.638199] multipathd 26041 0.000000 1 0 0.000000 1.981301 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.649537] multipathd 26347 0.000000 1 0 0.000000 2.626227 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.660871] multipathd 27527 0.000000 1 0 0.000000 1.823465 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.672209] multipathd 27778 0.000000 1 0 0.000000 1.896813 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.683544] multipathd 27840 0.000000 1 0 0.000000 1.678341 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.694880] multipathd 30895 0.000000 1 0 0.000000 2.776960 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.706216] multipathd 31227 0.000000 1 0 0.000000 1.762740 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.717551] kdmwork-253:0 12455 83923442.244577 868138865 120 0.000000 31839835.966567 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.728915] kdmflush 17976 8063.073481 2 100 0.000000 0.012063 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.740252] bioset 17978 8063.077182 2 100 0.000000 0.010699 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.751586] bioset 17998 8063.089536 2 100 0.000000 0.011210 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.762924] kdmflush 18001 8063.102640 2 100 0.000000 0.011330 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.774258] bioset 18002 8063.106829 2 100 0.000000 0.010250 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.785594] kdmflush 18007 8063.120115 2 100 0.000000 0.011642 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.796931] bioset 18008 8063.124783 2 100 0.000000 0.010790 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.808268] kdmflush 18012 8063.137339 2 100 0.000000 0.010709 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.819602] bioset 18013 8063.141267 2 100 0.000000 0.009909 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.830943] kdmflush 18065 8099.933520 2 100 0.000000 0.013005 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.842276] bioset 18066 8099.937741 2 100 0.000000 0.012663 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.853613] kdmflush 18132 8099.952214 2 100 0.000000 0.015199 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.864948] bioset 18134 8099.957194 2 100 0.000000 0.012844 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.876296] kdmflush 18439 8180.764215 2 100 0.000000 0.008246 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.887628] kdmflush 18485 8305.495531 2 100 0.000000 0.008787 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.898966] bioset 18505 8342.314975 2 100 0.000000 0.013095 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.910302] kdmflush 18523 8354.906216 2 100 0.000000 0.009037 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.921638] kdmflush 18541 8366.920634 2 100 0.000000 0.013755 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.932972] bioset 18542 8366.924905 2 100 0.000000 0.012473 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.944309] kdmflush 18551 8366.936484 2 100 0.000000 0.012994 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.955644] bioset 18553 8366.940877 2 100 0.000000 0.012464 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.966980] kdmflush 18560 8366.952806 2 100 0.000000 0.013014 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.978316] bioset 18561 8366.956637 2 100 0.000000 0.013003 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592654.989651] kdmflush 18570 8366.968647 2 100 0.000000 0.012294 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.000988] bioset 18573 8366.972658 2 100 0.000000 0.012293 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.012324] kdmflush 18582 8366.984366 2 100 0.000000 0.012453 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.023661] bioset 18585 8366.988517 2 100 0.000000 0.012845 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.034997] kdmflush 18593 8367.000899 2 100 0.000000 0.013736 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.046332] bioset 18595 8367.005161 2 100 0.000000 0.012824 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.057670] kdmflush 18661 8442.580325 2 100 0.000000 0.012554 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.069006] bioset 18663 8466.807654 2 100 0.000000 0.008506 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.080342] kdmflush 18691 8504.500125 2 100 0.000000 0.015739 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.091680] bioset 18814 8576.586551 2 100 0.000000 0.012654 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.103015] kdmflush 19042 8698.939019 2 100 0.000000 0.009328 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.114349] bioset 19045 8710.945799 2 100 0.000000 0.007955 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.125689] bioset 19373 8927.123060 2 100 0.000000 0.015930 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.137024] kdmflush 19662 9030.480451 2 100 0.000000 0.011211 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.148359] bioset 19663 9030.484295 2 100 0.000000 0.012614 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.159698] bioset 20669 9555.650155 2 100 0.000000 0.008085 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.171036] kdmflush 25377 11939.179606 2 100 0.000000 0.010971 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.182367] bioset 25378 11939.183997 2 100 0.000000 0.012163 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.193705] bioset 27525 13368.214151 2 100 0.000000 0.014066 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.205064] ptlrpcd_03_09 64850 83922989.273024 3568942 120 0.000000 114914.967586 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.216412] ldlm_cn03_000 66892 83923448.731275 26390578 120 0.000000 595336.165032 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.227750] ll_ost_io03_000 66970 83923186.046791 109940775 120 0.000000 6321761.143907 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.239082] ll_ost_io03_002 66974 83923431.327715 50480559 120 0.000000 3121885.640984 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.250436] ll_ost_io03_005 68374 83923276.369587 73023010 120 0.000000 4312498.597885 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.261776] ll_ost_io03_011 68397 83923442.310808 70382340 120 0.000000 4284256.659008 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.273116] ll_ost03_013 68533 83923521.902087 18625639 120 0.000000 540995.678673 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.284454] ll_ost03_017 68541 83923525.744063 14508321 120 0.000000 446488.861629 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.295801] ll_ost03_034 68602 82758605.896528 22720719 120 0.000000 631286.335236 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.307150] ll_ost_io03_025 68772 83923414.920903 64805983 120 0.000000 3871648.120061 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.318496] ll_ost_io03_033 68869 83923407.766242 97779249 120 0.000000 5347832.117066 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.329835] ll_ost_io03_042 68890 83923331.469043 43592795 120 0.000000 2825310.928432 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.341172] ll_ost_create03 69524 82968315.995864 3465626 120 0.000000 58999.715028 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.352516] ldlm_cn03_008 74194 83923527.363271 8227093 120 0.000000 194159.838374 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.363851] ll_ost_io03_046 97769 83923445.452885 113906577 120 0.000000 7276627.179214 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.375193] ldlm_bl_41 100122 83923526.577973 6109663 120 0.000000 83662.871404 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.386645] ll_ost_io03_048 84741 83923331.803819 137522073 120 0.000000 8208325.284280 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.397978] ll_ost_io03_052 84745 83923268.506323 67474211 120 0.000000 4151624.129150 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.409314] ll_ost_io03_057 87194 83923431.262290 52891974 120 0.000000 3026369.130272 0.000000 3 / [Tue Mar 17 18:02:50 2020][8592655.420669] ll_ost_io03_077 40772 83923538.819148 29658008 120 0.000000 1674547.102019 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.432011] ll_ost_io03_085 40800 83923440.037599 95492342 120 0.000000 5764642.234312 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.443357] ll_ost_io03_096 40839 83923385.170802 57574871 120 0.000000 3668580.838269 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.454696] ll_ost_io03_101 40849 83922987.499113 95465164 120 0.000000 5412685.904454 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.466030] ll_ost_io03_104 40857 83923266.681002 111814026 120 0.000000 6112774.177588 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.477365] ll_ost_io03_110 40863 83922999.461582 118900356 120 0.000000 6869389.622163 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.488700] ll_ost_io03_111 40864 83923437.820954 38996620 120 0.000000 2264550.077953 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.500049] ll_ost03_077 70324 82758605.896017 9927542 120 0.000000 313907.708094 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.511392] ll_ost03_106 125739 83923527.221951 17547537 120 0.000000 478371.659067 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.522824] kworker/47:0 45049 83923526.668024 150462 120 0.000000 2463.525883 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.534157] kworker/47:1 56815 82899959.218504 2 120 0.000000 0.010840 0.000000 3 / [Tue Mar 17 18:02:51 2020][8592655.545581] [Tue Mar 17 18:02:51 2020][8592655.547455] INFO: rcu_sched self-detected stall on CPU [Tue Mar 17 18:02:51 2020][8592655.547476] INFO: rcu_sched detected stalls on CPUs/tasks: { 6} (detected by 20, t=0 jiffies, g=803325448, c=803325447, q=1274043) [Tue Mar 17 18:02:51 2020][8592655.547477] Task dump for CPU 0: [Tue Mar 17 18:02:51 2020][8592655.547480] swapper/0 R running task 0 0 0 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547480] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547487] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547489] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547491] [] ? native_safe_halt+0xb/0x20 [Tue Mar 17 18:02:51 2020][8592655.547493] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547496] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547498] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547500] [] ? rest_init+0x77/0x80 [Tue Mar 17 18:02:51 2020][8592655.547503] [] ? start_kernel+0x44b/0x46c [Tue Mar 17 18:02:51 2020][8592655.547505] [] ? repair_env_string+0x5c/0x5c [Tue Mar 17 18:02:51 2020][8592655.547507] [] ? early_idt_handler_array+0x120/0x120 [Tue Mar 17 18:02:51 2020][8592655.547508] [] ? x86_64_start_reservations+0x24/0x26 [Tue Mar 17 18:02:51 2020][8592655.547510] [] ? x86_64_start_kernel+0x154/0x177 [Tue Mar 17 18:02:51 2020][8592655.547511] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547512] Task dump for CPU 1: [Tue Mar 17 18:02:51 2020][8592655.547515] swapper/1 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547518] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547520] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547521] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547522] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547523] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547525] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547527] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547528] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547529] Task dump for CPU 2: [Tue Mar 17 18:02:51 2020][8592655.547531] rs:main Q:Reg R running task 0 50893 1 0x00000080 [Tue Mar 17 18:02:51 2020][8592655.547531] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547534] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:02:51 2020][8592655.547536] [] schedule+0x29/0x70 [Tue Mar 17 18:02:51 2020][8592655.547538] [] futex_wait_queue_me+0xc6/0x130 [Tue Mar 17 18:02:51 2020][8592655.547540] [] futex_wait+0x17b/0x280 [Tue Mar 17 18:02:51 2020][8592655.547542] [] ? generic_file_aio_write+0x77/0xa0 [Tue Mar 17 18:02:51 2020][8592655.547544] [] ? get_futex_key+0x1c8/0x2c0 [Tue Mar 17 18:02:51 2020][8592655.547546] [] ? futex_wake+0x6a/0x180 [Tue Mar 17 18:02:51 2020][8592655.547550] [] ? sockfd_lookup_light+0x20/0x80 [Tue Mar 17 18:02:51 2020][8592655.547554] [] ? vfs_write+0xc0/0x1f0 [Tue Mar 17 18:02:51 2020][8592655.547557] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 18:02:51 2020][8592655.547559] [] ? system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:51 2020][8592655.547560] Task dump for CPU 3: [Tue Mar 17 18:02:51 2020][8592655.547562] swapper/3 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547563] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547565] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547566] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547567] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547569] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547570] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547572] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547574] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547575] Task dump for CPU 4: [Tue Mar 17 18:02:51 2020][8592655.547577] swapper/4 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547577] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547579] [] ? tick_nohz_stop_sched_tick+0x7c/0x380 [Tue Mar 17 18:02:51 2020][8592655.547580] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547581] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547582] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547584] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547585] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547586] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547586] Task dump for CPU 5: [Tue Mar 17 18:02:51 2020][8592655.547588] swapper/5 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547589] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547591] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547593] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547596] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547598] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547599] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547600] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547601] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547602] Task dump for CPU 6: [Tue Mar 17 18:02:51 2020][8592655.547604] swapper/6 R running task 0 0 1 0x00000008 [Tue Mar 17 18:02:51 2020][8592655.547604] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547606] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547607] [] ? native_safe_halt+0xb/0x20 [Tue Mar 17 18:02:51 2020][8592655.547608] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547610] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547611] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547612] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547615] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547617] Task dump for CPU 7: [Tue Mar 17 18:02:51 2020][8592655.547619] swapper/7 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547619] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547621] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547622] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547623] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547624] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547625] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547627] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547628] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547628] Task dump for CPU 8: [Tue Mar 17 18:02:51 2020][8592655.547630] lnetctl R running task 0 11793 11738 0x00000080 [Tue Mar 17 18:02:51 2020][8592655.547631] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547647] [] ? LNetCtl+0x5b7/0xe50 [lnet] [Tue Mar 17 18:02:51 2020][8592655.547650] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:02:51 2020][8592655.547658] [] ? LNetNIInit+0x7e/0xc10 [lnet] [Tue Mar 17 18:02:51 2020][8592655.547660] [] ? mutex_lock+0x12/0x2f [Tue Mar 17 18:02:51 2020][8592655.547668] [] ? LNetNIFini+0x39/0x110 [lnet] [Tue Mar 17 18:02:51 2020][8592655.547678] [] ? lnet_ioctl+0x6b/0x270 [lnet] [Tue Mar 17 18:02:51 2020][8592655.547681] [] ? __kmalloc+0x1eb/0x230 [Tue Mar 17 18:02:51 2020][8592655.547689] [] ? libcfs_ioctl_getdata+0x22c/0x770 [libcfs] [Tue Mar 17 18:02:51 2020][8592655.547696] [] ? libcfs_ioctl+0x4b/0x4c0 [libcfs] [Tue Mar 17 18:02:51 2020][8592655.547702] [] ? libcfs_psdev_ioctl+0x67/0xf0 [libcfs] [Tue Mar 17 18:02:51 2020][8592655.547705] [] ? do_vfs_ioctl+0x3a0/0x5a0 [Tue Mar 17 18:02:51 2020][8592655.547712] [] ? libcfs_ioctl+0x1a2/0x4c0 [libcfs] [Tue Mar 17 18:02:51 2020][8592655.547714] [] ? SyS_ioctl+0xa1/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547715] [] ? __audit_syscall_exit+0x1e6/0x280 [Tue Mar 17 18:02:51 2020][8592655.547717] [] ? auditsys+0x14/0x46 [Tue Mar 17 18:02:51 2020][8592655.547718] Task dump for CPU 9: [Tue Mar 17 18:02:51 2020][8592655.547720] swapper/9 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547720] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547722] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547723] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547725] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547726] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547727] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547730] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547732] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547733] Task dump for CPU 10: [Tue Mar 17 18:02:51 2020][8592655.547735] swapper/10 R running task 0 0 1 0x00800008 [Tue Mar 17 18:02:51 2020][8592655.547735] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547737] [] ? __schedule+0x534/0x860 [Tue Mar 17 18:02:51 2020][8592655.547738] [] ? update_ts_time_stats+0x58/0x90 [Tue Mar 17 18:02:51 2020][8592655.547740] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:02:51 2020][8592655.547741] [] cpu_startup_entry+0x18a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547743] [] start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547744] [] start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547745] Task dump for CPU 11: [Tue Mar 17 18:02:51 2020][8592655.547747] swapper/11 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547747] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547749] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547750] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547752] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547755] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547756] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547758] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547759] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547759] Task dump for CPU 12: [Tue Mar 17 18:02:51 2020][8592655.547761] swapper/12 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547761] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547762] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547763] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547765] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547766] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547767] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:51 2020][8592655.547768] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:51 2020][8592655.547769] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:51 2020][8592655.547770] Task dump for CPU 13: [Tue Mar 17 18:02:51 2020][8592655.547771] swapper/13 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:51 2020][8592655.547773] Call Trace: [Tue Mar 17 18:02:51 2020][8592655.547776] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:51 2020][8592655.547777] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:51 2020][8592655.547778] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547779] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:51 2020][8592655.547780] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547782] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547783] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547783] Task dump for CPU 14: [Tue Mar 17 18:02:52 2020][8592655.547786] journalctl R running task 0 102169 102158 0x00000080 [Tue Mar 17 18:02:52 2020][8592655.547786] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547788] [] ? __schedule+0x42a/0x860 [Tue Mar 17 18:02:52 2020][8592655.547790] [] ? ktime_get_ts64+0x52/0xf0 [Tue Mar 17 18:02:52 2020][8592655.547791] [] schedule+0x29/0x70 [Tue Mar 17 18:02:52 2020][8592655.547794] [] schedule_hrtimeout_range_clock+0x12d/0x150 [Tue Mar 17 18:02:52 2020][8592655.547797] [] ? bit_wait+0x50/0x50 [Tue Mar 17 18:02:52 2020][8592655.547798] [] ? __delayacct_blkio_end+0x34/0x60 [Tue Mar 17 18:02:52 2020][8592655.547801] [] ? list_del+0xd/0x30 [Tue Mar 17 18:02:52 2020][8592655.547803] [] ? remove_wait_queue+0x31/0x40 [Tue Mar 17 18:02:52 2020][8592655.547804] [] ? poll_freewait+0x47/0xa0 [Tue Mar 17 18:02:52 2020][8592655.547806] [] ? do_sys_poll+0xd2/0x590 [Tue Mar 17 18:02:52 2020][8592655.547809] [] ? security_inode_permission+0x22/0x30 [Tue Mar 17 18:02:52 2020][8592655.547810] [] ? __inode_permission+0x52/0xd0 [Tue Mar 17 18:02:52 2020][8592655.547811] [] ? inode_permission+0x18/0x50 [Tue Mar 17 18:02:52 2020][8592655.547812] [] ? link_path_walk+0x81/0x8b0 [Tue Mar 17 18:02:52 2020][8592655.547815] [] ? __pollwait+0xf0/0xf0 [Tue Mar 17 18:02:52 2020][8592655.547818] [] ? legitimize_mnt+0x16/0x40 [Tue Mar 17 18:02:52 2020][8592655.547820] [] ? path_lookupat+0x7a/0x8b0 [Tue Mar 17 18:02:52 2020][8592655.547822] [] ? kmem_cache_alloc+0x35/0x1f0 [Tue Mar 17 18:02:52 2020][8592655.547823] [] ? getname_flags+0x4f/0x1a0 [Tue Mar 17 18:02:52 2020][8592655.547824] [] ? putname+0x3d/0x60 [Tue Mar 17 18:02:52 2020][8592655.547826] [] ? user_path_at_empty+0x72/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547828] [] ? __check_object_size+0x1ca/0x250 [Tue Mar 17 18:02:52 2020][8592655.547829] [] ? list_del+0xd/0x30 [Tue Mar 17 18:02:52 2020][8592655.547831] [] ? remove_wait_queue+0x31/0x40 [Tue Mar 17 18:02:52 2020][8592655.547833] [] ? inotify_read+0x2ef/0x420 [Tue Mar 17 18:02:52 2020][8592655.547834] [] ? abort_exclusive_wait+0xa0/0xa0 [Tue Mar 17 18:02:52 2020][8592655.547836] [] ? SyS_write+0x7f/0xf0 [Tue Mar 17 18:02:52 2020][8592655.547837] [] ? system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:52 2020][8592655.547838] Task dump for CPU 15: [Tue Mar 17 18:02:52 2020][8592655.547841] swapper/15 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547843] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547845] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547846] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547847] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547848] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547850] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547851] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547852] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547853] Task dump for CPU 16: [Tue Mar 17 18:02:52 2020][8592655.547854] swapper/16 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547854] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547856] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547857] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547859] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547860] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547862] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547865] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547867] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547867] Task dump for CPU 17: [Tue Mar 17 18:02:52 2020][8592655.547869] swapper/17 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547869] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547871] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547872] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547873] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547874] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547875] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547877] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547878] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547878] Task dump for CPU 18: [Tue Mar 17 18:02:52 2020][8592655.547880] swapper/18 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547880] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547882] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547883] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547886] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547888] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547889] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547891] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547892] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547892] Task dump for CPU 19: [Tue Mar 17 18:02:52 2020][8592655.547894] swapper/19 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547894] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547896] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547897] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547898] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547899] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547901] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547902] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547903] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547903] Task dump for CPU 20: [Tue Mar 17 18:02:52 2020][8592655.547905] swapper/20 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547907] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547910] [] sched_show_task+0xa8/0x110 [Tue Mar 17 18:02:52 2020][8592655.547912] [] dump_cpu_task+0x39/0x70 [Tue Mar 17 18:02:52 2020][8592655.547913] [] rcu_dump_cpu_stacks+0x90/0xd0 [Tue Mar 17 18:02:52 2020][8592655.547914] [] rcu_check_callbacks+0x6f2/0x730 [Tue Mar 17 18:02:52 2020][8592655.547916] [] ? tick_sched_do_timer+0x50/0x50 [Tue Mar 17 18:02:52 2020][8592655.547918] [] update_process_times+0x46/0x80 [Tue Mar 17 18:02:52 2020][8592655.547919] [] tick_sched_handle+0x30/0x70 [Tue Mar 17 18:02:52 2020][8592655.547921] [] tick_sched_timer+0x39/0x80 [Tue Mar 17 18:02:52 2020][8592655.547923] [] __hrtimer_run_queues+0xf3/0x270 [Tue Mar 17 18:02:52 2020][8592655.547925] [] hrtimer_interrupt+0xaf/0x1d0 [Tue Mar 17 18:02:52 2020][8592655.547927] [] local_apic_timer_interrupt+0x3b/0x60 [Tue Mar 17 18:02:52 2020][8592655.547929] [] smp_apic_timer_interrupt+0x43/0x60 [Tue Mar 17 18:02:52 2020][8592655.547932] [] apic_timer_interrupt+0x162/0x170 [Tue Mar 17 18:02:52 2020][8592655.547935] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547936] [] ? native_safe_halt+0xb/0x20 [Tue Mar 17 18:02:52 2020][8592655.547938] [] default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547939] [] arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547940] [] cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547941] [] start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547943] [] start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547943] Task dump for CPU 21: [Tue Mar 17 18:02:52 2020][8592655.547945] swapper/21 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547945] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547947] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547948] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547950] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547953] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547954] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547955] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547956] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547957] Task dump for CPU 22: [Tue Mar 17 18:02:52 2020][8592655.547959] systemd-journal R running task 0 5729 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547959] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547961] [] ? do_sys_open+0x10c/0x240 [Tue Mar 17 18:02:52 2020][8592655.547963] [] SyS_open+0x1e/0x20 [Tue Mar 17 18:02:52 2020][8592655.547965] [] system_call_fastpath+0x22/0x27 [Tue Mar 17 18:02:52 2020][8592655.547965] Task dump for CPU 23: [Tue Mar 17 18:02:52 2020][8592655.547967] swapper/23 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547967] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547969] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547970] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547971] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547973] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547976] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547977] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547978] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547979] Task dump for CPU 24: [Tue Mar 17 18:02:52 2020][8592655.547980] swapper/24 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547981] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547982] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547983] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547985] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547986] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.547987] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.547988] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.547989] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.547990] Task dump for CPU 25: [Tue Mar 17 18:02:52 2020][8592655.547991] swapper/25 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.547992] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.547993] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.547996] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.547999] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548000] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548001] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.548002] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.548003] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.548004] Task dump for CPU 26: [Tue Mar 17 18:02:52 2020][8592655.548006] swapper/26 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.548006] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.548008] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.548009] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.548011] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548012] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548013] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.548014] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.548015] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.548016] Task dump for CPU 27: [Tue Mar 17 18:02:52 2020][8592655.548019] swapper/27 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:52 2020][8592655.548020] Call Trace: [Tue Mar 17 18:02:52 2020][8592655.548022] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:52 2020][8592655.548023] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:52 2020][8592655.548024] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548025] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:52 2020][8592655.548026] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:52 2020][8592655.548027] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:52 2020][8592655.548028] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:52 2020][8592655.548029] Task dump for CPU 28: [Tue Mar 17 18:02:52 2020][8592655.548030] swapper/28 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548031] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548032] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548033] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548034] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548035] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548037] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548039] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548042] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548042] Task dump for CPU 29: [Tue Mar 17 18:02:53 2020][8592655.548044] swapper/29 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548044] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548046] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548047] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548049] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548050] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548051] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548052] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548053] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548054] Task dump for CPU 30: [Tue Mar 17 18:02:53 2020][8592655.548055] swapper/30 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548055] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548058] [] ? tick_nohz_stop_sched_tick+0x7c/0x380 [Tue Mar 17 18:02:53 2020][8592655.548060] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548063] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548064] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548065] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548067] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548068] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548068] Task dump for CPU 31: [Tue Mar 17 18:02:53 2020][8592655.548070] swapper/31 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548070] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548072] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548073] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548074] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548075] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548076] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548078] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548079] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548079] Task dump for CPU 32: [Tue Mar 17 18:02:53 2020][8592655.548081] swapper/32 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548081] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548083] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548085] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548088] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548089] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548090] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548092] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548093] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548093] Task dump for CPU 33: [Tue Mar 17 18:02:53 2020][8592655.548095] swapper/33 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548095] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548097] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548098] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548099] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548100] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548101] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548103] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548104] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548104] Task dump for CPU 34: [Tue Mar 17 18:02:53 2020][8592655.548106] swapper/34 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548106] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548108] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548111] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548114] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548115] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548116] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548117] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548118] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548119] Task dump for CPU 35: [Tue Mar 17 18:02:53 2020][8592655.548121] raid-check R running task 0 14331 64016 0x00000080 [Tue Mar 17 18:02:53 2020][8592655.548121] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548122] Task dump for CPU 36: [Tue Mar 17 18:02:53 2020][8592655.548124] swapper/36 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548124] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548125] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548126] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548128] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548129] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548130] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548131] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548134] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548136] Task dump for CPU 37: [Tue Mar 17 18:02:53 2020][8592655.548137] swapper/37 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548138] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548139] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548140] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548142] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548143] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548144] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548145] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548146] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548147] Task dump for CPU 38: [Tue Mar 17 18:02:53 2020][8592655.548148] swapper/38 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548149] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548150] [] ? __schedule+0x652/0x860 [Tue Mar 17 18:02:53 2020][8592655.548152] [] ? activate_task+0x4c/0xa0 [Tue Mar 17 18:02:53 2020][8592655.548154] [] ? ttwu_do_activate+0x6f/0x80 [Tue Mar 17 18:02:53 2020][8592655.548155] [] schedule_preempt_disabled+0x29/0x70 [Tue Mar 17 18:02:53 2020][8592655.548157] [] cpu_startup_entry+0x18a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548160] [] start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548161] [] start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548162] Task dump for CPU 39: [Tue Mar 17 18:02:53 2020][8592655.548163] swapper/39 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548164] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548165] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548167] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548168] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548169] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548170] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548171] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548172] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548173] Task dump for CPU 40: [Tue Mar 17 18:02:53 2020][8592655.548174] swapper/40 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548174] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548176] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548178] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548181] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548183] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548184] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548185] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548186] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548187] Task dump for CPU 41: [Tue Mar 17 18:02:53 2020][8592655.548188] swapper/41 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548189] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548190] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548191] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548193] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548194] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548195] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548196] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548197] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548198] Task dump for CPU 42: [Tue Mar 17 18:02:53 2020][8592655.548201] swapper/42 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548202] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548203] [] ? tick_nohz_stop_sched_tick+0x7c/0x380 [Tue Mar 17 18:02:53 2020][8592655.548205] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548206] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548207] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548208] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548209] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548210] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548211] Task dump for CPU 43: [Tue Mar 17 18:02:53 2020][8592655.548213] swapper/43 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548213] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548215] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548216] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548217] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548219] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548222] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548223] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548224] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548225] Task dump for CPU 44: [Tue Mar 17 18:02:53 2020][8592655.548226] swapper/44 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548226] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548228] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548229] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548230] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548231] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548232] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548234] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548235] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548235] Task dump for CPU 45: [Tue Mar 17 18:02:53 2020][8592655.548237] swapper/45 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548237] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548239] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548241] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548244] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548245] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548247] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548248] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548249] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548249] Task dump for CPU 46: [Tue Mar 17 18:02:53 2020][8592655.548251] swapper/46 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548251] Call Trace: [Tue Mar 17 18:02:53 2020][8592655.548253] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:53 2020][8592655.548254] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:53 2020][8592655.548255] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548256] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:53 2020][8592655.548257] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:53 2020][8592655.548259] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:53 2020][8592655.548260] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:53 2020][8592655.548260] Task dump for CPU 47: [Tue Mar 17 18:02:53 2020][8592655.548262] swapper/47 R running task 0 0 1 0x00000000 [Tue Mar 17 18:02:53 2020][8592655.548264] Call Trace: [Tue Mar 17 18:02:54 2020][8592655.548267] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:54 2020][8592655.548268] [] ? rcu_eqs_enter_common.isra.30+0x3d/0xe0 [Tue Mar 17 18:02:54 2020][8592655.548270] [] ? default_idle+0x1e/0xc0 [Tue Mar 17 18:02:54 2020][8592655.548271] [] ? arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:54 2020][8592655.548272] [] ? cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:54 2020][8592655.548273] [] ? start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:54 2020][8592655.548274] [] ? start_cpu+0x5/0x14 [Tue Mar 17 18:02:54 2020][8592657.644476] LustreError: dumping log to /tmp/lustre-log.1584493373.68617 [Tue Mar 17 18:02:54 2020][8592658.478629] { 6} (t=2933 jiffies g=803325448 c=803325447 q=1484465) [Tue Mar 17 18:02:54 2020][8592658.485326] Task dump for CPU 6: [Tue Mar 17 18:02:54 2020][8592658.488733] swapper/6 R running task 0 0 1 0x00000008 [Tue Mar 17 18:02:54 2020][8592658.496003] Call Trace: [Tue Mar 17 18:02:54 2020][8592658.498633] [] sched_show_task+0xa8/0x110 [Tue Mar 17 18:02:54 2020][8592658.505106] [] dump_cpu_task+0x39/0x70 [Tue Mar 17 18:02:54 2020][8592658.510680] [] rcu_dump_cpu_stacks+0x90/0xd0 [Tue Mar 17 18:02:54 2020][8592658.516771] [] rcu_check_callbacks+0x442/0x730 [Tue Mar 17 18:02:54 2020][8592658.523039] [] ? tick_sched_do_timer+0x50/0x50 [Tue Mar 17 18:02:54 2020][8592658.529305] [] update_process_times+0x46/0x80 [Tue Mar 17 18:02:54 2020][8592658.535482] [] tick_sched_handle+0x30/0x70 [Tue Mar 17 18:02:54 2020][8592658.541401] [] tick_sched_timer+0x39/0x80 [Tue Mar 17 18:02:54 2020][8592658.547237] [] __hrtimer_run_queues+0xf3/0x270 [Tue Mar 17 18:02:54 2020][8592658.553499] [] hrtimer_interrupt+0xaf/0x1d0 [Tue Mar 17 18:02:54 2020][8592658.559509] [] local_apic_timer_interrupt+0x3b/0x60 [Tue Mar 17 18:02:54 2020][8592658.566209] [] smp_apic_timer_interrupt+0x43/0x60 [Tue Mar 17 18:02:54 2020][8592658.572732] [] apic_timer_interrupt+0x162/0x170 [Tue Mar 17 18:02:54 2020][8592658.579082] [] ? __cpuidle_text_start+0x8/0x8 [Tue Mar 17 18:02:54 2020][8592658.585896] [] ? native_safe_halt+0xb/0x20 [Tue Mar 17 18:02:54 2020][8592658.591815] [] default_idle+0x1e/0xc0 [Tue Mar 17 18:02:54 2020][8592658.597303] [] arch_cpu_idle+0x20/0xc0 [Tue Mar 17 18:02:54 2020][8592658.602875] [] cpu_startup_entry+0x14a/0x1e0 [Tue Mar 17 18:02:54 2020][8592658.608968] [] start_secondary+0x1f7/0x270 [Tue Mar 17 18:02:54 2020][8592658.614887] [] start_cpu+0x5/0x14 [Tue Mar 17 18:02:54 2020][8592658.704246] LNet: Service thread pid 68624 completed after 265.32s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). [Tue Mar 17 18:02:54 2020][8592658.720580] LNet: Skipped 8 previous similar messages [Tue Mar 17 18:02:55 2020][8592659.399352] LNet: Service thread pid 73710 completed after 283.25s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). [Tue Mar 17 18:02:55 2020][8592659.415686] LNet: Skipped 45 previous similar messages [Tue Mar 17 18:02:56 2020][8592660.608620] LNet: Service thread pid 68721 completed after 287.09s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). [Tue Mar 17 18:02:56 2020][8592660.624949] LNet: Skipped 37 previous similar messages [Tue Mar 17 18:02:58 2020][8592662.637694] LNet: Service thread pid 40847 completed after 274.43s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). [Tue Mar 17 18:02:58 2020][8592662.654024] LNet: Skipped 161 previous similar messages [Tue Mar 17 18:03:02 2020][8592666.695298] LNet: Service thread pid 97765 completed after 291.25s. This indicates the system was overloaded (too many service threads, or there were not enough hardware resources). [Tue Mar 17 18:03:02 2020][8592666.711859] LNet: Skipped 261 previous similar messages [Tue Mar 17 18:06:02 2020][8592847.373122] LNetError: 11485:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:06:02 2020][8592847.385304] LNetError: 11485:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 174 previous similar messages [Tue Mar 17 18:09:43 2020][8593067.436798] LNetError: 13721:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.203@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:09:43 2020][8593067.448277] LNetError: 13721:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 352 previous similar messages [Tue Mar 17 18:11:56 2020][8593200.606234] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.235@o2ib7: 0 seconds [Tue Mar 17 18:11:56 2020][8593200.616495] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 214 previous similar messages [Tue Mar 17 18:16:07 2020][8593451.455710] LNetError: 118397:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:16:07 2020][8593451.467971] LNetError: 118397:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 160 previous similar messages [Tue Mar 17 18:19:47 2020][8593671.479387] LNetError: 118397:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.212@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:19:47 2020][8593671.490955] LNetError: 118397:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 341 previous similar messages [Tue Mar 17 18:22:01 2020][8593805.608152] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.235@o2ib7: 0 seconds [Tue Mar 17 18:22:01 2020][8593805.618499] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 57 previous similar messages [Tue Mar 17 18:26:07 2020][8594051.534345] LNetError: 14841:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:26:07 2020][8594051.546632] LNetError: 14841:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 154 previous similar messages [Tue Mar 17 18:29:47 2020][8594271.541997] LNetError: 15235:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.210@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:29:47 2020][8594271.553480] LNetError: 15235:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 354 previous similar messages [Tue Mar 17 18:32:03 2020][8594407.609995] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.218@o2ib7: 1 seconds [Tue Mar 17 18:32:03 2020][8594407.620252] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 116 previous similar messages [Tue Mar 17 18:36:07 2020][8594651.576309] LNetError: 15429:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:36:07 2020][8594651.588483] LNetError: 15429:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 170 previous similar messages [Tue Mar 17 18:39:52 2020][8594876.604937] LNetError: 15429:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.202@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:39:52 2020][8594876.616431] LNetError: 15429:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 364 previous similar messages [Tue Mar 17 18:42:08 2020][8595012.611849] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.233@o2ib7: 2 seconds [Tue Mar 17 18:42:08 2020][8595012.622188] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 326 previous similar messages [Tue Mar 17 18:46:07 2020][8595251.651184] LNetError: 15753:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:46:07 2020][8595251.663359] LNetError: 15753:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 164 previous similar messages [Tue Mar 17 18:49:52 2020][8595476.673926] LNetError: 16004:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.202@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:49:52 2020][8595476.685409] LNetError: 16004:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 355 previous similar messages [Tue Mar 17 18:52:12 2020][8595616.613505] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.239@o2ib7: 1 seconds [Tue Mar 17 18:52:12 2020][8595616.623845] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 487 previous similar messages [Tue Mar 17 18:56:07 2020][8595851.733894] LNetError: 16246:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 18:56:07 2020][8595851.746129] LNetError: 16246:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 200 previous similar messages [Tue Mar 17 18:59:57 2020][8596081.742245] LNetError: 16004:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.211@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 18:59:57 2020][8596081.753731] LNetError: 16004:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 353 previous similar messages [Tue Mar 17 19:02:13 2020][8596217.614557] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.219@o2ib7: 0 seconds [Tue Mar 17 19:02:13 2020][8596217.624815] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 420 previous similar messages [Tue Mar 17 19:06:07 2020][8596451.770500] LNetError: 16513:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:06:07 2020][8596451.782676] LNetError: 16513:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 232 previous similar messages [Tue Mar 17 19:09:57 2020][8596681.792324] LNetError: 16780:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.212@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:09:57 2020][8596681.803800] LNetError: 16780:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 345 previous similar messages [Tue Mar 17 19:12:15 2020][8596819.615604] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.233@o2ib7: 1 seconds [Tue Mar 17 19:12:15 2020][8596819.625948] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 398 previous similar messages [Tue Mar 17 19:16:07 2020][8597051.827403] LNetError: 16780:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:16:07 2020][8597051.839582] LNetError: 16780:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 219 previous similar messages [Tue Mar 17 19:19:57 2020][8597281.862451] LNetError: 17162:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.211@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:19:57 2020][8597281.873936] LNetError: 17162:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 366 previous similar messages [Tue Mar 17 19:22:17 2020][8597421.616633] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.233@o2ib7: 0 seconds [Tue Mar 17 19:22:17 2020][8597421.626894] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 269 previous similar messages [Tue Mar 17 19:26:07 2020][8597651.913050] LNetError: 17220:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:26:07 2020][8597651.925222] LNetError: 17220:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 231 previous similar messages [Tue Mar 17 19:29:57 2020][8597881.915497] LNetError: 17391:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.211@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:29:57 2020][8597881.926986] LNetError: 17391:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 352 previous similar messages [Tue Mar 17 19:32:28 2020][8598032.617636] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.232@o2ib7: 0 seconds [Tue Mar 17 19:32:28 2020][8598032.627984] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 435 previous similar messages [Tue Mar 17 19:36:07 2020][8598251.952339] LNetError: 17674:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:36:07 2020][8598251.964524] LNetError: 17674:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 219 previous similar messages [Tue Mar 17 19:39:57 2020][8598481.981582] LNetError: 17962:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.211@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:39:57 2020][8598481.993078] LNetError: 17962:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 366 previous similar messages [Tue Mar 17 19:42:49 2020][8598653.618689] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.236@o2ib7: 0 seconds [Tue Mar 17 19:42:49 2020][8598653.628945] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 208 previous similar messages [Tue Mar 17 19:46:07 2020][8598852.043875] LNetError: 17962:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:46:07 2020][8598852.056062] LNetError: 17962:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 163 previous similar messages [Tue Mar 17 19:49:57 2020][8599082.072694] LNetError: 18513:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.204@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:49:57 2020][8599082.084180] LNetError: 18513:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 386 previous similar messages [Tue Mar 17 19:52:52 2020][8599256.619735] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.234@o2ib7: 0 seconds [Tue Mar 17 19:52:52 2020][8599256.629993] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 45 previous similar messages [Tue Mar 17 19:56:07 2020][8599452.102040] LNetError: 18876:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 19:56:07 2020][8599452.114237] LNetError: 18876:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 160 previous similar messages [Tue Mar 17 19:59:57 2020][8599682.114273] LNetError: 18876:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.204@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 19:59:57 2020][8599682.125755] LNetError: 18876:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 335 previous similar messages [Tue Mar 17 20:03:01 2020][8599865.621008] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.217@o2ib7: 0 seconds [Tue Mar 17 20:03:01 2020][8599865.631265] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 187 previous similar messages [Tue Mar 17 20:06:07 2020][8600052.155508] LNetError: 19085:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:06:07 2020][8600052.167689] LNetError: 19085:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 198 previous similar messages [Tue Mar 17 20:09:57 2020][8600282.195729] LNetError: 19085:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.204@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 20:09:57 2020][8600282.207214] LNetError: 19085:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 376 previous similar messages [Tue Mar 17 20:13:02 2020][8600466.622405] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.226@o2ib7: 0 seconds [Tue Mar 17 20:13:02 2020][8600466.632660] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 170 previous similar messages [Tue Mar 17 20:16:07 2020][8600652.244490] LNetError: 19507:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:16:07 2020][8600652.256662] LNetError: 19507:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 195 previous similar messages [Tue Mar 17 20:19:57 2020][8600882.289731] LNetError: 19762:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.212@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 20:19:57 2020][8600882.301213] LNetError: 19762:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 367 previous similar messages [Tue Mar 17 20:23:12 2020][8601076.624010] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.238@o2ib7: 0 seconds [Tue Mar 17 20:23:12 2020][8601076.634269] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 182 previous similar messages [Tue Mar 17 20:26:07 2020][8601252.333119] LNetError: 19963:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:26:07 2020][8601252.345290] LNetError: 19963:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 208 previous similar messages [Tue Mar 17 20:29:57 2020][8601482.367007] LNetError: 19963:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.203@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 20:29:57 2020][8601482.378481] LNetError: 19963:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 387 previous similar messages [Tue Mar 17 20:33:33 2020][8601697.625792] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.235@o2ib7: 0 seconds [Tue Mar 17 20:33:33 2020][8601697.636138] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 245 previous similar messages [Tue Mar 17 20:36:07 2020][8601852.431998] LNetError: 19963:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:36:07 2020][8601852.444188] LNetError: 19963:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 204 previous similar messages [Tue Mar 17 20:39:58 2020][8602082.457098] LNetError: 20534:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.210@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 20:39:58 2020][8602082.468571] LNetError: 20534:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 381 previous similar messages [Tue Mar 17 20:43:42 2020][8602306.627488] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.227@o2ib7: 3 seconds [Tue Mar 17 20:43:42 2020][8602306.637751] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 113 previous similar messages [Tue Mar 17 20:46:12 2020][8602456.513464] LNetError: 20734:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:46:12 2020][8602456.525638] LNetError: 20734:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 166 previous similar messages [Tue Mar 17 20:50:02 2020][8602686.576632] LNetError: 20734:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.210@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 20:50:02 2020][8602686.588125] LNetError: 20734:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 399 previous similar messages [Tue Mar 17 20:53:45 2020][8602909.629298] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.234@o2ib7: 0 seconds [Tue Mar 17 20:53:45 2020][8602909.639557] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 172 previous similar messages [Tue Mar 17 20:56:12 2020][8603056.629737] LNetError: 64781:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 20:56:12 2020][8603056.641910] LNetError: 64781:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 189 previous similar messages [Tue Mar 17 21:00:02 2020][8603286.665096] LNetError: 21083:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.210@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:00:02 2020][8603286.676841] LNetError: 21083:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 371 previous similar messages [Tue Mar 17 21:03:49 2020][8603513.631121] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.219@o2ib7: 0 seconds [Tue Mar 17 21:03:49 2020][8603513.641379] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 710 previous similar messages [Tue Mar 17 21:06:12 2020][8603656.715260] LNetError: 4643:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:06:12 2020][8603656.727351] LNetError: 4643:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 189 previous similar messages [Tue Mar 17 21:10:02 2020][8603886.751044] LNetError: 21578:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.209@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:10:02 2020][8603886.762529] LNetError: 21578:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 393 previous similar messages [Tue Mar 17 21:13:50 2020][8604114.632715] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.233@o2ib7: 0 seconds [Tue Mar 17 21:13:50 2020][8604114.642973] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 457 previous similar messages [Tue Mar 17 21:16:12 2020][8604256.798179] LNetError: 21869:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:16:12 2020][8604256.810361] LNetError: 21869:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 217 previous similar messages [Tue Mar 17 21:20:02 2020][8604486.831145] LNetError: 22105:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.204@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:20:02 2020][8604486.842624] LNetError: 22105:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 388 previous similar messages [Tue Mar 17 21:23:52 2020][8604716.634124] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.10.236@o2ib7: 0 seconds [Tue Mar 17 21:23:52 2020][8604716.644466] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 220 previous similar messages [Tue Mar 17 21:26:12 2020][8604856.874424] LNetError: 22296:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:26:12 2020][8604856.886593] LNetError: 22296:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 214 previous similar messages [Tue Mar 17 21:30:02 2020][8605086.899128] LNetError: 22515:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.203@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:30:02 2020][8605086.910641] LNetError: 22515:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 375 previous similar messages [Tue Mar 17 21:33:57 2020][8605321.635409] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.217@o2ib7: 0 seconds [Tue Mar 17 21:33:57 2020][8605321.645663] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 242 previous similar messages [Tue Mar 17 21:36:12 2020][8605456.957429] LNetError: 22515:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:36:12 2020][8605456.969602] LNetError: 22515:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 218 previous similar messages [Tue Mar 17 21:40:07 2020][8605691.984334] LNetError: 22515:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.209@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:40:07 2020][8605691.995836] LNetError: 22515:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 394 previous similar messages [Tue Mar 17 21:44:17 2020][8605941.636675] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.219@o2ib7: 0 seconds [Tue Mar 17 21:44:17 2020][8605941.646932] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 234 previous similar messages [Tue Mar 17 21:46:12 2020][8606057.019594] LNetError: 23012:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:46:12 2020][8606057.031824] LNetError: 23012:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 204 previous similar messages [Tue Mar 17 21:50:07 2020][8606292.032491] LNetError: 23012:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.211@o2ib7 rejected: consumer defined fatal error [Tue Mar 17 21:50:07 2020][8606292.044005] LNetError: 23012:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) Skipped 346 previous similar messages [Tue Mar 17 21:54:18 2020][8606542.637974] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Timed out tx for 10.0.2.225@o2ib7: 6 seconds [Tue Mar 17 21:54:18 2020][8606542.648235] LNet: 64781:0:(o2iblnd_cb.c:3396:kiblnd_check_conns()) Skipped 324 previous similar messages [Tue Mar 17 21:56:17 2020][8606662.047089] LNetError: 23012:0:(lib-msg.c:485:lnet_handle_local_failure()) ni 10.0.10.113@o2ib7 added to recovery queue. Health = 900 [Tue Mar 17 21:56:17 2020][8606662.059265] LNetError: 23012:0:(lib-msg.c:485:lnet_handle_local_failure()) Skipped 209 previous similar messages [Tue Mar 17 22:00:07 2020][8606892.057900] LNetError: 23621:0:(o2iblnd_cb.c:2923:kiblnd_rejected()) 10.0.10.204@o2ib7 rejected: consumer defined fatal error