Console [hyperion-agb21] log at 2014-01-07 18:00:00 PST. 2014-01-07 18:24:45 LustreError: 7113:0:(ofd_grant.c:163:ofd_grant_sanity_check()) ofd_obd_disconnect: tot_granted 36749312000 != fo_tot_granted 37856608256 2014-01-07 18:24:45 LustreError: 7119:0:(ofd_grant.c:163:ofd_grant_sanity_check()) ofd_obd_disconnect: tot_granted 36715757568 != fo_tot_granted 36749312000 Console [hyperion-agb21] joined by on pts/1 at 01-07 18:53. 2014-01-07 18:54:04 SysRq : Show State 2014-01-07 18:54:04 task PC stack pid father 2014-01-07 18:54:04 init S 0000000000000009 0 1 0 0x00000000 2014-01-07 18:54:04 ffff8808344b5908 0000000000000086 0000000000000000 ffff8808344b58cc 2014-01-07 18:54:04 0000000000000000 ffff88083fe82200 ffff88085c4f6740 0000000000000400 2014-01-07 18:54:04 ffff881034f6c5f8 ffff8808344b5fd8 000000000000fb88 ffff881034f6c5f8 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:04 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:04 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:04 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:04 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:04 [] do_select+0x57c/0x6c0 2014-01-07 18:54:04 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:04 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:04 [] ? pollwake+0x0/0x60 2014-01-07 18:54:04 [] ? pollwake+0x0/0x60 2014-01-07 18:54:04 [] ? pollwake+0x0/0x60 2014-01-07 18:54:04 [] ? pollwake+0x0/0x60 2014-01-07 18:54:04 [] ? pollwake+0x0/0x60 2014-01-07 18:54:04 [] ? d_lookup+0x35/0x60 2014-01-07 18:54:04 [] ? d_hash_and_lookup+0x83/0xb0 2014-01-07 18:54:04 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:54:04 [] ? security_task_wait+0x16/0x20 2014-01-07 18:54:04 [] ? wait_consider_task+0x9d/0xb20 2014-01-07 18:54:04 [] ? remove_wait_queue+0x3c/0x50 2014-01-07 18:54:04 [] ? do_wait+0x17f/0x240 2014-01-07 18:54:04 [] sys_select+0x47/0x110 2014-01-07 18:54:04 [] ? child_wait_callback+0x0/0x70 2014-01-07 18:54:04 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:04 kthreadd S 0000000000000000 0 2 0 0x00000000 2014-01-07 18:54:04 ffff8808344bbed0 0000000000000046 ffff8808344bbf40 ffffffff8100c062 2014-01-07 18:54:04 ffff8808344b9500 ffff8808344b9500 ffffffff81eafb88 ffff881003dc1d18 2014-01-07 18:54:04 ffff8808344b9ab8 ffff8808344bbfd8 000000000000fb88 ffff8808344b9ab8 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ? kernel_thread+0x82/0xe0 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] kthreadd+0x1b5/0x1c0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthreadd+0x0/0x1c0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 migration/0 S 0000000000000000 0 3 2 0x00000000 2014-01-07 18:54:04 ffff8808344ffe70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:04 ffff8808344ffdf0 ffff8800bd4b6080 ffff88085c436740 0000000000000000 2014-01-07 18:54:04 ffff8808344b9058 ffff8808344fffd8 000000000000fb88 ffff8808344b9058 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:04 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:04 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 ksoftirqd/0 S 0000000000000000 0 4 2 0x00000000 2014-01-07 18:54:04 ffff880834501ea0 0000000000000046 0000000000000000 ffff880834501e64 2014-01-07 18:54:04 0000000000000000 ffff88083fe81000 ffff88085c4f6740 00000000000005fe 2014-01-07 18:54:04 ffff8808344b85f8 ffff880834501fd8 000000000000fb88 ffff8808344b85f8 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:04 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 migration/0 S 0000000000000000 0 5 2 0x00000000 2014-01-07 18:54:04 ffff880834505df0 0000000000000046 0000000000000000 ffff88083287bd28 2014-01-07 18:54:04 0000000000000000 0000000000000000 ffff880834505d70 ffffffff81063422 2014-01-07 18:54:04 ffff880834503af8 ffff880834505fd8 000000000000fb88 ffff880834503af8 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:04 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:04 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:04 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:04 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:04 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 watchdog/0 S 0000000000000000 0 6 2 0x00000000 2014-01-07 18:54:04 ffff880834509ea0 0000000000000046 0000000000000000 ffff880834509e64 2014-01-07 18:54:04 ffff880800000000 ffff88083fe81000 ffff880044696740 00000000000002af 2014-01-07 18:54:04 ffff880834503098 ffff880834509fd8 000000000000fb88 ffff880834503098 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] watchdog+0x9a/0xd0 2014-01-07 18:54:04 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 migration/1 R running task 0 7 2 0x00000000 2014-01-07 18:54:04 ffff88083450be70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:04 ffff88083450bdf0 ffff880e7a873500 ffff88085c436740 0000000000000000 2014-01-07 18:54:04 ffff880834502638 ffff88083450bfd8 000000000000fb88 ffff880834502638 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:04 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:04 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 migration/1 S 0000000000000001 0 8 2 0x00000000 2014-01-07 18:54:04 ffff88083450fdf0 0000000000000046 0000000000000000 ffff88083450fdb4 2014-01-07 18:54:04 0000000000000000 ffff88083fe81200 ffff880044616740 0000000000000200 2014-01-07 18:54:04 ffff88083450dab8 ffff88083450ffd8 000000000000fb88 ffff88083450dab8 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:04 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:04 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:04 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:04 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 ksoftirqd/1 S 0000000000000001 0 9 2 0x00000000 2014-01-07 18:54:04 ffff880834103ea0 0000000000000046 0000000000000000 ffff880834103e64 2014-01-07 18:54:04 ffff880800000000 ffff88083fe81200 ffff880044676740 0000000000000400 2014-01-07 18:54:04 ffff88083450d058 ffff880834103fd8 000000000000fb88 ffff88083450d058 2014-01-07 18:54:04 Call Trace: 2014-01-07 18:54:04 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:04 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:04 [] kthread+0x96/0xa0 2014-01-07 18:54:04 [] child_rip+0xa/0x20 2014-01-07 18:54:04 [] ? kthread+0x0/0xa0 2014-01-07 18:54:04 [] ? child_rip+0x0/0x20 2014-01-07 18:54:04 watchdog/1 S 0000000000000001 0 10 2 0x00000000 2014-01-07 18:54:04 ffff88083410dea0 0000000000000046 0000000000000000 ffff88083410de64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81200 ffff8800446d6740 0000000000000207 2014-01-07 18:54:05 ffff88083410baf8 ffff88083410dfd8 000000000000fb88 ffff88083410baf8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] watchdog+0x9a/0xd0 2014-01-07 18:54:05 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/2 S 0000000000000002 0 11 2 0x00000000 2014-01-07 18:54:05 ffff88083410fe70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:05 ffff88083410fdf0 ffff8800bd002ae0 ffff880044676740 0000000000000000 2014-01-07 18:54:05 ffff88083410b098 ffff88083410ffd8 000000000000fb88 ffff88083410b098 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:05 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:05 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/2 S 0000000000000002 0 12 2 0x00000000 2014-01-07 18:54:05 ffff880834113df0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:05 0000000000000000 00000000ffffffff ffff880834112000 0000000000000000 2014-01-07 18:54:05 ffff88083410a638 ffff880834113fd8 000000000000fb88 ffff88083410a638 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:05 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:05 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:05 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:05 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 ksoftirqd/2 S 0000000000000002 0 13 2 0x00000000 2014-01-07 18:54:05 ffff88083413bea0 0000000000000046 0000000000000000 ffff88083413be64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:05 ffff880834139ab8 ffff88083413bfd8 000000000000fb88 ffff880834139ab8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:05 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 watchdog/2 S 0000000000000002 0 14 2 0x00000000 2014-01-07 18:54:05 ffff880834141ea0 0000000000000046 0000000000000000 ffff880834141e64 2014-01-07 18:54:05 0000000000000000 ffff88083fe81400 ffff88085c4b6740 0000000000000400 2014-01-07 18:54:05 ffff880834139058 ffff880834141fd8 000000000000fb88 ffff880834139058 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] watchdog+0x9a/0xd0 2014-01-07 18:54:05 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/3 S 0000000000000003 0 15 2 0x00000000 2014-01-07 18:54:05 ffff880834143e70 0000000000000046 ffff880834143dd0 ffff8800bd18f500 2014-01-07 18:54:05 ffff880834143e00 ffffffff81062a2e ffff88085c436740 ffff8800bd18f500 2014-01-07 18:54:05 ffff8808341385f8 ffff880834143fd8 000000000000fb88 ffff8808341385f8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:05 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:05 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/3 S 0000000000000003 0 16 2 0x00000000 2014-01-07 18:54:05 ffff880834149df0 0000000000000046 0000000000000000 ffff880834149db4 2014-01-07 18:54:05 0000000000000000 ffff88083fe81600 ffff880044616740 0000000000000200 2014-01-07 18:54:05 ffff880834147af8 ffff880834149fd8 000000000000fb88 ffff880834147af8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:05 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:05 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:05 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:05 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 ksoftirqd/3 S 0000000000000003 0 17 2 0x00000000 2014-01-07 18:54:05 ffff880834151ea0 0000000000000046 0000000000000000 ffff880834151e64 2014-01-07 18:54:05 0000000000000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:05 ffff880834147098 ffff880834151fd8 000000000000fb88 ffff880834147098 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:05 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 watchdog/3 S 0000000000000003 0 18 2 0x00000000 2014-01-07 18:54:05 ffff880834155ea0 0000000000000046 0000000000000000 ffff880834155e64 2014-01-07 18:54:05 0000000000000000 ffff88083fe81600 ffff880044656740 0000000000000207 2014-01-07 18:54:05 ffff880834153ab8 ffff880834155fd8 000000000000fb88 ffff880834153ab8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] watchdog+0x9a/0xd0 2014-01-07 18:54:05 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/4 S 0000000000000004 0 19 2 0x00000000 2014-01-07 18:54:05 ffff880834157e70 0000000000000046 ffff880834157dd0 ffff8800bb826040 2014-01-07 18:54:05 ffff880834157e00 ffffffff81062a2e ffff8800446b6740 ffff8800bb826040 2014-01-07 18:54:05 ffff880834153058 ffff880834157fd8 000000000000fb88 ffff880834153058 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:05 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:05 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/4 S 0000000000000004 0 20 2 0x00000000 2014-01-07 18:54:05 ffff88083415bdf0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:05 0000000000000000 00000000ffffffff ffff88083415a000 0000000000000000 2014-01-07 18:54:05 ffff8808341525f8 ffff88083415bfd8 000000000000fb88 ffff8808341525f8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:05 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:05 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:05 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:05 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 ksoftirqd/4 S 0000000000000004 0 21 2 0x00000000 2014-01-07 18:54:05 ffff880834181ea0 0000000000000046 0000000000000000 ffff880834181e64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81800 ffff8800446b6740 0000000000000400 2014-01-07 18:54:05 ffff88083417faf8 ffff880834181fd8 000000000000fb88 ffff88083417faf8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:05 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 watchdog/4 S 0000000000000004 0 22 2 0x00000000 2014-01-07 18:54:05 ffff880834189ea0 0000000000000046 0000000000000000 ffff880834189e64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81800 ffff88085c4b6740 00000000000007fe 2014-01-07 18:54:05 ffff88083417f098 ffff880834189fd8 000000000000fb88 ffff88083417f098 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] watchdog+0x9a/0xd0 2014-01-07 18:54:05 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/5 S 0000000000000005 0 23 2 0x00000000 2014-01-07 18:54:05 ffff88083418be70 0000000000000046 ffff88083418bdd0 ffff8800bb826040 2014-01-07 18:54:05 ffff88083418be00 ffffffff81062a2e ffff88085c436740 ffff8800bb826040 2014-01-07 18:54:05 ffff88083417e638 ffff88083418bfd8 000000000000fb88 ffff88083417e638 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:05 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:05 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 migration/5 S 0000000000000005 0 24 2 0x00000000 2014-01-07 18:54:05 ffff88083418fdf0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:05 0000000000000000 00000000ffffffff ffff88083418e000 0000000000000000 2014-01-07 18:54:05 ffff88083418dab8 ffff88083418ffd8 000000000000fb88 ffff88083418dab8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:05 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:05 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:05 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:05 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 ksoftirqd/5 S 0000000000000005 0 25 2 0x00000000 2014-01-07 18:54:05 ffff880834193ea0 0000000000000046 ffff880834193e68 ffff880834193e64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81a00 ffff880044636740 0000000000000207 2014-01-07 18:54:05 ffff88083418d058 ffff880834193fd8 000000000000fb88 ffff88083418d058 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:05 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:05 [] kthread+0x96/0xa0 2014-01-07 18:54:05 [] child_rip+0xa/0x20 2014-01-07 18:54:05 [] ? kthread+0x0/0xa0 2014-01-07 18:54:05 [] ? child_rip+0x0/0x20 2014-01-07 18:54:05 watchdog/5 S 0000000000000005 0 26 2 0x00000000 2014-01-07 18:54:05 ffff88083419bea0 0000000000000046 0000000000000000 ffff88083419be64 2014-01-07 18:54:05 ffff880800000000 ffff88083fe81a00 ffff8800446d6740 0000000000000400 2014-01-07 18:54:05 ffff880834199af8 ffff88083419bfd8 000000000000fb88 ffff880834199af8 2014-01-07 18:54:05 Call Trace: 2014-01-07 18:54:05 [] watchdog+0x9a/0xd0 2014-01-07 18:54:05 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/6 S 0000000000000006 0 27 2 0x00000000 2014-01-07 18:54:06 ffff88083419de70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:06 ffff88083419ddf0 ffff8800bcd7c080 ffff88085c436740 0000000000000000 2014-01-07 18:54:06 ffff880834199098 ffff88083419dfd8 000000000000fb88 ffff880834199098 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:06 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:06 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/6 S 0000000000000006 0 28 2 0x00000000 2014-01-07 18:54:06 ffff8808341a3df0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:06 0000000000000000 00000000ffffffff ffff8808341a2000 0000000000000005 2014-01-07 18:54:06 ffff880834198638 ffff8808341a3fd8 000000000000fb88 ffff880834198638 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:06 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:06 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:06 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:06 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 ksoftirqd/6 S 0000000000000006 0 29 2 0x00000000 2014-01-07 18:54:06 ffff8808341cbea0 0000000000000046 0000000000000000 ffff8808341cbe64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe81c00 ffff88085c436740 0000000000000400 2014-01-07 18:54:06 ffff8808341c9ab8 ffff8808341cbfd8 000000000000fb88 ffff8808341c9ab8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:06 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 watchdog/6 S 0000000000000006 0 30 2 0x00000000 2014-01-07 18:54:06 ffff8808341d1ea0 0000000000000046 0000000000000000 ffff8808341d1e64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe81c00 ffff88085c456740 0000000000000200 2014-01-07 18:54:06 ffff8808341c9058 ffff8808341d1fd8 000000000000fb88 ffff8808341c9058 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] watchdog+0x9a/0xd0 2014-01-07 18:54:06 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/7 S 0000000000000007 0 31 2 0x00000000 2014-01-07 18:54:06 ffff8808341d3e70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:06 ffff8808341d3df0 ffff88007f9c4080 ffff88085c436740 0000000000000000 2014-01-07 18:54:06 ffff8808341c85f8 ffff8808341d3fd8 000000000000fb88 ffff8808341c85f8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:06 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:06 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/7 S 0000000000000007 0 32 2 0x00000000 2014-01-07 18:54:06 ffff8808341d7df0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:06 0000000000000000 00000000ffffffff ffff8808341d6000 0000000000000000 2014-01-07 18:54:06 ffff8808341d5af8 ffff8808341d7fd8 000000000000fb88 ffff8808341d5af8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:06 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:06 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:06 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:06 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 ksoftirqd/7 S 0000000000000007 0 33 2 0x00000000 2014-01-07 18:54:06 ffff8808341dbea0 0000000000000046 0000000000000000 ffff8808341dbe64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe81e00 ffff880044656740 00000000000002af 2014-01-07 18:54:06 ffff8808341d5098 ffff8808341dbfd8 000000000000fb88 ffff8808341d5098 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:06 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 watchdog/7 S 0000000000000007 0 34 2 0x00000000 2014-01-07 18:54:06 ffff8808341e5ea0 0000000000000046 0000000000000000 ffff8808341e5e64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe81e00 ffff8800446b6740 0000000000000400 2014-01-07 18:54:06 ffff8808341e3ab8 ffff8808341e5fd8 000000000000fb88 ffff8808341e3ab8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] watchdog+0x9a/0xd0 2014-01-07 18:54:06 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/8 S 0000000000000008 0 35 2 0x00000000 2014-01-07 18:54:06 ffff8808341e7e70 0000000000000046 ffff8808341e7df0 ffffffff81065deb 2014-01-07 18:54:06 ffff8808341e7df0 000000007c5e3540 ffff88085c436740 ffff88085c436740 2014-01-07 18:54:06 ffff8808341e3058 ffff8808341e7fd8 000000000000fb88 ffff8808341e3058 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? enqueue_task_fair+0xfb/0x100 2014-01-07 18:54:06 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:06 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/8 S 0000000000000008 0 36 2 0x00000000 2014-01-07 18:54:06 ffff8808341ebdf0 0000000000000046 0000000000000000 ffff8808341ebdb4 2014-01-07 18:54:06 0000000000000000 ffff88083fe82000 ffff880044616740 0000000000000716 2014-01-07 18:54:06 ffff8808341e25f8 ffff8808341ebfd8 000000000000fb88 ffff8808341e25f8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:06 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:06 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:06 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:06 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 ksoftirqd/8 S 0000000000000008 0 37 2 0x00000000 2014-01-07 18:54:06 ffff8808341efea0 0000000000000046 0000000000000000 ffff8808341efe64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe82000 ffff88085c436740 0000000000000207 2014-01-07 18:54:06 ffff8808341edaf8 ffff8808341effd8 000000000000fb88 ffff8808341edaf8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:06 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 watchdog/8 S 0000000000000008 0 38 2 0x00000000 2014-01-07 18:54:06 ffff880834213ea0 0000000000000046 0000000000000000 ffff880834213e64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe82000 ffff8800446b6740 00000000000007fe 2014-01-07 18:54:06 ffff8808341ed098 ffff880834213fd8 000000000000fb88 ffff8808341ed098 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] watchdog+0x9a/0xd0 2014-01-07 18:54:06 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/9 S 0000000000000009 0 39 2 0x00000000 2014-01-07 18:54:06 ffff880834215e70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:06 ffff880834215df0 ffff88007f2ae040 ffff880044636740 0000000000000000 2014-01-07 18:54:06 ffff8808341ec638 ffff880834215fd8 000000000000fb88 ffff8808341ec638 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:06 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:06 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/9 S 0000000000000009 0 40 2 0x00000000 2014-01-07 18:54:06 ffff880834219df0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:06 0000000000000000 00000000ffffffff ffff880834218000 0000000000000000 2014-01-07 18:54:06 ffff880834217ab8 ffff880834219fd8 000000000000fb88 ffff880834217ab8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:06 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:06 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:06 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:06 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 ksoftirqd/9 S 0000000000000009 0 41 2 0x00000000 2014-01-07 18:54:06 ffff88083421fea0 0000000000000046 ffff88083421fe68 ffff88083421fe64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe82200 ffff880044636740 0000000000000207 2014-01-07 18:54:06 ffff880834217058 ffff88083421ffd8 000000000000fb88 ffff880834217058 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:06 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 watchdog/9 S 0000000000000009 0 42 2 0x00000000 2014-01-07 18:54:06 ffff880834223ea0 0000000000000046 0000000000000000 ffff880834223e64 2014-01-07 18:54:06 ffff880800000000 ffff88083fe82200 ffff88085c476740 0000000000000207 2014-01-07 18:54:06 ffff880834221af8 ffff880834223fd8 000000000000fb88 ffff880834221af8 2014-01-07 18:54:06 Call Trace: 2014-01-07 18:54:06 [] watchdog+0x9a/0xd0 2014-01-07 18:54:06 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:06 [] kthread+0x96/0xa0 2014-01-07 18:54:06 [] child_rip+0xa/0x20 2014-01-07 18:54:06 [] ? kthread+0x0/0xa0 2014-01-07 18:54:06 [] ? child_rip+0x0/0x20 2014-01-07 18:54:06 migration/10 S 000000000000000a 0 43 2 0x00000000 2014-01-07 18:54:07 ffff880834225e70 0000000000000046 0000000000000000 ffffffff81065d54 2014-01-07 18:54:07 ffff880834225df0 ffff8800bc88c080 ffff880044636740 0000000000000000 2014-01-07 18:54:07 ffff880834221098 ffff880834225fd8 000000000000fb88 ffff880834221098 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:07 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:07 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/10 S 000000000000000a 0 44 2 0x00000000 2014-01-07 18:54:07 ffff880834229df0 0000000000000046 0000000000000000 ffff880834229db4 2014-01-07 18:54:07 0000000000000000 ffff88083fe82400 ffff880044616740 0000000000000716 2014-01-07 18:54:07 ffff880834220638 ffff880834229fd8 000000000000fb88 ffff880834220638 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:07 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:07 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:07 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:07 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 ksoftirqd/10 S 000000000000000a 0 45 2 0x00000000 2014-01-07 18:54:07 ffff88083422dea0 0000000000000046 0000000000000000 ffffffff8100bb8e 2014-01-07 18:54:07 ffff88083422dea0 0000000000000000 0000000000000000 00000000ffffffff 2014-01-07 18:54:07 ffff88083422bab8 ffff88083422dfd8 000000000000fb88 ffff88083422bab8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:07 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:07 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 watchdog/10 S 000000000000000a 0 46 2 0x00000000 2014-01-07 18:54:07 ffff880834233ea0 0000000000000046 0000000000000000 ffff880834233e64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82400 ffff880044696740 00000000000001ff 2014-01-07 18:54:07 ffff88083422b058 ffff880834233fd8 000000000000fb88 ffff88083422b058 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] watchdog+0x9a/0xd0 2014-01-07 18:54:07 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/11 S 000000000000000b 0 47 2 0x00000000 2014-01-07 18:54:07 ffff880834237e70 0000000000000046 ffff880834237dd0 ffff8800bb826040 2014-01-07 18:54:07 ffff880834237e00 ffffffff81062a2e ffff88085c496740 ffff8800bb826040 2014-01-07 18:54:07 ffff88083422a5f8 ffff880834237fd8 000000000000fb88 ffff88083422a5f8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:07 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:07 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/11 S 000000000000000b 0 48 2 0x00000000 2014-01-07 18:54:07 ffff88083423bdf0 0000000000000046 0000000000000000 ffff88083423bdb4 2014-01-07 18:54:07 0000000000000000 ffff88083fe82600 ffff880044616740 0000000000000716 2014-01-07 18:54:07 ffff880834239af8 ffff88083423bfd8 000000000000fb88 ffff880834239af8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:07 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:07 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:07 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:07 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 ksoftirqd/11 S 000000000000000b 0 49 2 0x00000000 2014-01-07 18:54:07 ffff88083423fea0 0000000000000046 ffff88083423fe68 ffff88083423fe64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82600 ffff88085c436740 0000000000000480 2014-01-07 18:54:07 ffff880834239098 ffff88083423ffd8 000000000000fb88 ffff880834239098 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:07 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 watchdog/11 S 000000000000000b 0 50 2 0x00000000 2014-01-07 18:54:07 ffff880834243ea0 0000000000000046 0000000000000000 ffff880834243e64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82600 ffff88085c416740 0000000000000400 2014-01-07 18:54:07 ffff880834241ab8 ffff880834243fd8 000000000000fb88 ffff880834241ab8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] watchdog+0x9a/0xd0 2014-01-07 18:54:07 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/12 S 000000000000000c 0 51 2 0x00000000 2014-01-07 18:54:07 ffff880834245e70 0000000000000046 ffff880834245dd0 ffff8800bb826040 2014-01-07 18:54:07 ffff880834245e00 ffffffff81062a2e ffff88085c4b6740 ffff8800bb826040 2014-01-07 18:54:07 ffff880834241058 ffff880834245fd8 000000000000fb88 ffff880834241058 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:07 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:07 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/12 S 000000000000000c 0 52 2 0x00000000 2014-01-07 18:54:07 ffff880834249df0 0000000000000046 0000000000000000 ffff880834249db4 2014-01-07 18:54:07 0000000000000000 ffff88083fe82800 ffff880044616740 0000000000000716 2014-01-07 18:54:07 ffff8808342405f8 ffff880834249fd8 000000000000fb88 ffff8808342405f8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:07 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:07 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:07 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:07 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 ksoftirqd/12 S 000000000000000c 0 53 2 0x00000000 2014-01-07 18:54:07 ffff88083424fea0 0000000000000046 ffff88083424fe68 ffff88083424fe64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82800 ffff88085c4d6740 0000000000000400 2014-01-07 18:54:07 ffff88083424daf8 ffff88083424ffd8 000000000000fb88 ffff88083424daf8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:07 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 watchdog/12 S 000000000000000c 0 54 2 0x00000000 2014-01-07 18:54:07 ffff880834255ea0 0000000000000046 0000000000000000 ffff880834255e64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82800 ffff8800446f6740 00000000000005fe 2014-01-07 18:54:07 ffff88083424d098 ffff880834255fd8 000000000000fb88 ffff88083424d098 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] watchdog+0x9a/0xd0 2014-01-07 18:54:07 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/13 S 000000000000000d 0 55 2 0x00000000 2014-01-07 18:54:07 ffff880834257e70 0000000000000046 ffff880834257dd0 ffff881007228080 2014-01-07 18:54:07 ffff880834257e00 ffffffff81062a2e ffff88085c4d6740 ffff881007228080 2014-01-07 18:54:07 ffff88083424c638 ffff880834257fd8 000000000000fb88 ffff88083424c638 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:07 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:07 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/13 S 000000000000000d 0 56 2 0x00000000 2014-01-07 18:54:07 ffff88083425bdf0 0000000000000046 0000000000000000 ffff88103093de38 2014-01-07 18:54:07 0000000000000000 0000000000000000 ffff88083425bd70 ffffffff81063422 2014-01-07 18:54:07 ffff880834259ab8 ffff88083425bfd8 000000000000fb88 ffff880834259ab8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:07 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:07 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:07 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:07 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:07 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 ksoftirqd/13 S 000000000000000d 0 57 2 0x00000000 2014-01-07 18:54:07 ffff88083425fea0 0000000000000046 0000000000000000 ffff88083425fe64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82a00 ffff880044616740 00000000000007fe 2014-01-07 18:54:07 ffff880834259058 ffff88083425ffd8 000000000000fb88 ffff880834259058 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:07 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 watchdog/13 S 000000000000000d 0 58 2 0x00000000 2014-01-07 18:54:07 ffff880834263ea0 0000000000000046 0000000000000000 ffff880834263e64 2014-01-07 18:54:07 ffff880800000000 ffff88083fe82a00 ffff8800446b6740 00000000000007fe 2014-01-07 18:54:07 ffff880834261af8 ffff880834263fd8 000000000000fb88 ffff880834261af8 2014-01-07 18:54:07 Call Trace: 2014-01-07 18:54:07 [] watchdog+0x9a/0xd0 2014-01-07 18:54:07 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:07 [] kthread+0x96/0xa0 2014-01-07 18:54:07 [] child_rip+0xa/0x20 2014-01-07 18:54:07 [] ? kthread+0x0/0xa0 2014-01-07 18:54:07 [] ? child_rip+0x0/0x20 2014-01-07 18:54:07 migration/14 S 000000000000000e 0 59 2 0x00000000 2014-01-07 18:54:07 ffff880834267e70 0000000000000046 ffff880834267dd0 ffff881031b88ae0 2014-01-07 18:54:08 ffff880834267e00 ffffffff81062a2e ffff88085c456740 ffff881031b88ae0 2014-01-07 18:54:08 ffff880834261098 ffff880834267fd8 000000000000fb88 ffff880834261098 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:08 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:08 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 migration/14 S 000000000000000e 0 60 2 0x00000000 2014-01-07 18:54:08 ffff88083426bdf0 0000000000000046 0000000000000000 ffff88083426bdb4 2014-01-07 18:54:08 0000000000000000 ffff88083fe82c00 ffff880044616740 0000000000000716 2014-01-07 18:54:08 ffff880834260638 ffff88083426bfd8 000000000000fb88 ffff880834260638 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:08 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:08 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:08 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:08 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 ksoftirqd/14 S 000000000000000e 0 61 2 0x00000000 2014-01-07 18:54:08 ffff88083426fea0 0000000000000046 ffff88083426fe68 ffff88083426fe64 2014-01-07 18:54:08 ffff880800000000 ffff88083fe82c00 ffff88085c436740 0000000000000400 2014-01-07 18:54:08 ffff88083426dab8 ffff88083426ffd8 000000000000fb88 ffff88083426dab8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:08 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 watchdog/14 S 000000000000000e 0 62 2 0x00000000 2014-01-07 18:54:08 ffff880834275ea0 0000000000000046 0000000000000000 ffff880834275e64 2014-01-07 18:54:08 ffff880800000000 ffff88083fe82c00 ffff880044636740 00000000000001ff 2014-01-07 18:54:08 ffff88083426d058 ffff880834275fd8 000000000000fb88 ffff88083426d058 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] watchdog+0x9a/0xd0 2014-01-07 18:54:08 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 migration/15 S 000000000000000f 0 63 2 0x00000000 2014-01-07 18:54:08 ffff880834277e70 0000000000000046 ffff880834277dd0 ffff8800bb826040 2014-01-07 18:54:08 ffff880834277e00 ffffffff81062a2e ffff88085c436740 ffff8800bb826040 2014-01-07 18:54:08 ffff88083426c5f8 ffff880834277fd8 000000000000fb88 ffff88083426c5f8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? pull_task+0x4e/0x60 2014-01-07 18:54:08 [] migration_thread+0x265/0x2e0 2014-01-07 18:54:08 [] ? migration_thread+0x0/0x2e0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 migration/15 S 000000000000000f 0 64 2 0x00000000 2014-01-07 18:54:08 ffff88083427bdf0 0000000000000046 0000000000000000 ffff880804e3fe38 2014-01-07 18:54:08 0000000000000000 0000000000000000 ffff88083427bd70 ffffffff81063422 2014-01-07 18:54:08 ffff880834279af8 ffff88083427bfd8 000000000000fb88 ffff880834279af8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:08 [] ? stop_machine_cpu_stop+0x0/0xf0 2014-01-07 18:54:08 [] cpu_stopper_thread+0x125/0x1b0 2014-01-07 18:54:08 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:08 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:08 [] ? cpu_stopper_thread+0x0/0x1b0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 ksoftirqd/15 S 000000000000000f 0 65 2 0x00000000 2014-01-07 18:54:08 ffff88083427fea0 0000000000000046 ffff88083427fe68 ffff88083427fe64 2014-01-07 18:54:08 0000000000000000 ffff88083fe82e00 ffff88085c436740 0000000000000207 2014-01-07 18:54:08 ffff880834279098 ffff88083427ffd8 000000000000fb88 ffff880834279098 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ksoftirqd+0xd5/0x110 2014-01-07 18:54:08 [] ? ksoftirqd+0x0/0x110 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 watchdog/15 S 000000000000000f 0 66 2 0x00000000 2014-01-07 18:54:08 ffff880834285ea0 0000000000000046 0000000000000000 ffff880834285e64 2014-01-07 18:54:08 ffff880800000000 ffff88083fe82e00 ffff880044636740 0000000000000406 2014-01-07 18:54:08 ffff880834283ab8 ffff880834285fd8 000000000000fb88 ffff880834283ab8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] watchdog+0x9a/0xd0 2014-01-07 18:54:08 [] ? watchdog+0x0/0xd0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/0 S 0000000000000000 0 131 2 0x00000000 2014-01-07 18:54:08 ffff880833c71e30 0000000000000046 ffff880833c71d90 0000000000000282 2014-01-07 18:54:08 ffff880833c71de0 ffffffff81082134 ffff8808311f2020 0000000000000282 2014-01-07 18:54:08 ffff880833c6faf8 ffff880833c71fd8 000000000000fb88 ffff880833c6faf8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? mod_timer+0x144/0x220 2014-01-07 18:54:08 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:08 [] ? igb_watchdog_task+0x0/0x590 [igb] 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/1 S 0000000000000001 0 132 2 0x00000000 2014-01-07 18:54:08 ffff880833c97e30 0000000000000046 0000000000000000 ffff880044639480 2014-01-07 18:54:08 00000000ffffffff ffff8800bd48fe00 ffff880833c97dc0 0000000000000286 2014-01-07 18:54:08 ffff880833c6f098 ffff880833c97fd8 000000000000fb88 ffff880833c6f098 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? flush_to_ldisc+0x0/0x1b0 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/2 S 0000000000000002 0 133 2 0x00000000 2014-01-07 18:54:08 ffff880833c99e30 0000000000000046 0000000000000000 ffff880833c99df4 2014-01-07 18:54:08 0000000000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:08 ffff880833c6e638 ffff880833c99fd8 000000000000fb88 ffff880833c6e638 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/3 S 0000000000000003 0 134 2 0x00000000 2014-01-07 18:54:08 ffff880833c9de30 0000000000000046 0000000000000000 ffff880833c9ddf4 2014-01-07 18:54:08 0000000000000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:08 ffff880833c9bab8 ffff880833c9dfd8 000000000000fb88 ffff880833c9bab8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/4 S 0000000000000004 0 135 2 0x00000000 2014-01-07 18:54:08 ffff880833ca1e30 0000000000000046 0000000000000000 ffff880833ca1df4 2014-01-07 18:54:08 0000000000000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:08 ffff880833c9b058 ffff880833ca1fd8 000000000000fb88 ffff880833c9b058 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/5 S 0000000000000005 0 136 2 0x00000000 2014-01-07 18:54:08 ffff880833ca3e30 0000000000000046 00000000fffffffa ffff8800446b9480 2014-01-07 18:54:08 00000000ffffffff ffff8800446b3420 ffff880833ca3fd8 ffff8800446b9488 2014-01-07 18:54:08 ffff880833c9a5f8 ffff880833ca3fd8 000000000000fb88 ffff880833c9a5f8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/6 S 0000000000000006 0 137 2 0x00000000 2014-01-07 18:54:08 ffff880833ca7e30 0000000000000046 0000000000000000 ffff880833ca7df4 2014-01-07 18:54:08 0000000000000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:08 ffff880833ca5af8 ffff880833ca7fd8 000000000000fb88 ffff880833ca5af8 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/7 S 0000000000000007 0 138 2 0x00000000 2014-01-07 18:54:08 ffff880833cabe30 0000000000000046 0000000000000000 ffff880833cabdf4 2014-01-07 18:54:08 0000000000000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:54:08 ffff880833ca5098 ffff880833cabfd8 000000000000fb88 ffff880833ca5098 2014-01-07 18:54:08 Call Trace: 2014-01-07 18:54:08 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:08 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:08 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:08 [] kthread+0x96/0xa0 2014-01-07 18:54:08 [] child_rip+0xa/0x20 2014-01-07 18:54:08 [] ? kthread+0x0/0xa0 2014-01-07 18:54:08 [] ? child_rip+0x0/0x20 2014-01-07 18:54:08 events/8 S 0000000000000008 0 139 2 0x00000000 2014-01-07 18:54:08 ffff880833cafe30 0000000000000046 0000000000000000 ffff880833cafdf4 2014-01-07 18:54:08 0000000000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:08 ffff880833ca4638 ffff880833caffd8 000000000000fb88 ffff880833ca4638 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/9 S 0000000000000009 0 140 2 0x00000000 2014-01-07 18:54:09 ffff880833cb3e30 0000000000000046 0000000000000000 ffff880833cb3df4 2014-01-07 18:54:09 0000000000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833cb1ab8 ffff880833cb3fd8 000000000000fb88 ffff880833cb1ab8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/10 S 000000000000000a 0 141 2 0x00000000 2014-01-07 18:54:09 ffff880833cb7e30 0000000000000046 0000000000000002 ffff88085c459480 2014-01-07 18:54:09 00000000ffffffff ffff88085c453420 ffff880833cb7fd8 ffff88085c459488 2014-01-07 18:54:09 ffff880833cb1058 ffff880833cb7fd8 000000000000fb88 ffff880833cb1058 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/11 S 000000000000000b 0 142 2 0x00000000 2014-01-07 18:54:09 ffff880833cb9e30 0000000000000046 0000000000000000 ffff880833cb9df4 2014-01-07 18:54:09 0000000000000000 ffff88083fe82600 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833cb05f8 ffff880833cb9fd8 000000000000fb88 ffff880833cb05f8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/12 S 000000000000000c 0 143 2 0x00000000 2014-01-07 18:54:09 ffff880833cbde30 0000000000000046 0000000000000000 ffff880833cbddf4 2014-01-07 18:54:09 0000000000000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833cbbaf8 ffff880833cbdfd8 000000000000fb88 ffff880833cbbaf8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/13 S 000000000000000d 0 144 2 0x00000000 2014-01-07 18:54:09 ffff880833cc1e30 0000000000000046 0000000000000000 ffff880833cc1df4 2014-01-07 18:54:09 0000000000000000 ffff88083fe82a00 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833cbb098 ffff880833cc1fd8 000000000000fb88 ffff880833cbb098 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/14 S 000000000000000e 0 145 2 0x00000000 2014-01-07 18:54:09 ffff880833cc3e30 0000000000000046 0000000000000000 ffff88085c4d9480 2014-01-07 18:54:09 00000000ffffffff ffff88085c4d3420 ffff880833cc3fd8 ffff88085c4d9488 2014-01-07 18:54:09 ffff880833cba638 ffff880833cc3fd8 000000000000fb88 ffff880833cba638 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 events/15 S 000000000000000f 0 146 2 0x00000000 2014-01-07 18:54:09 ffff880833cc9e30 0000000000000046 0000000000000000 ffff880833cc9df4 2014-01-07 18:54:09 0000000000000000 ffff88083fe82e00 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833cc7ab8 ffff880833cc9fd8 000000000000fb88 ffff880833cc7ab8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? vmstat_update+0x0/0x40 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 cgroup S 0000000000000019 0 163 2 0x00000000 2014-01-07 18:54:09 ffff880833d05e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:09 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:09 ffff880833cfc638 ffff880833d05fd8 000000000000fb88 ffff880833cfc638 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 khelper S 0000000000000008 0 164 2 0x00000000 2014-01-07 18:54:09 ffff880833d09e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:09 ffff880833d09e30 ffff8807bf589d40 0000000000000000 0000000000000000 2014-01-07 18:54:09 ffff880833d07ab8 ffff880833d09fd8 000000000000fb88 ffff880833d07ab8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? __call_usermodehelper+0x0/0xd0 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 netns S 0000000000000019 0 165 2 0x00000000 2014-01-07 18:54:09 ffff880833d79e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:09 ffff880833d79db0 ffffffff81057310 ffff880833d06ad8 ffff88085c5367a8 2014-01-07 18:54:09 ffff880833d07058 ffff880833d79fd8 000000000000fb88 ffff880833d07058 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 async/mgr S 000000000000000b 0 166 2 0x00000000 2014-01-07 18:54:09 ffff880833d7be70 0000000000000046 ffff880833d7be38 ffff880833d7be34 2014-01-07 18:54:09 ffff880833d7bdf0 ffff88083fe82600 ffff88085c416740 00000000000006e2 2014-01-07 18:54:09 ffff880833d065f8 ffff880833d7bfd8 000000000000fb88 ffff880833d065f8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] async_manager_thread+0xc5/0x100 2014-01-07 18:54:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:09 [] ? async_manager_thread+0x0/0x100 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 pm S 0000000000000019 0 167 2 0x00000000 2014-01-07 18:54:09 ffff880833d91e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:09 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:09 ffff880833d7faf8 ffff880833d91fd8 000000000000fb88 ffff880833d7faf8 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:09 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 sync_supers S 0000000000000000 0 168 2 0x00000000 2014-01-07 18:54:09 ffff880833e33ec0 0000000000000046 0000000000000000 ffff880833e33e84 2014-01-07 18:54:09 ffff880800000000 ffff88083fe82a00 ffff880044616740 0000000000000400 2014-01-07 18:54:09 ffff880833d7f098 ffff880833e33fd8 000000000000fb88 ffff880833d7f098 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] ? bdi_sync_supers+0x0/0x60 2014-01-07 18:54:09 [] bdi_sync_supers+0x3b/0x60 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 bdi-default S 000000000000000a 0 169 2 0x00000000 2014-01-07 18:54:09 ffff880833e53dd0 0000000000000046 0000000000000000 ffff880833e53d94 2014-01-07 18:54:09 ffffffff00000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:54:09 ffff880833d7e638 ffff880833e53fd8 000000000000fb88 ffff880833d7e638 2014-01-07 18:54:09 Call Trace: 2014-01-07 18:54:09 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:09 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:09 [] bdi_forker_task+0x263/0x310 2014-01-07 18:54:09 [] ? bdi_forker_task+0x0/0x310 2014-01-07 18:54:09 [] kthread+0x96/0xa0 2014-01-07 18:54:09 [] child_rip+0xa/0x20 2014-01-07 18:54:09 [] ? kthread+0x0/0xa0 2014-01-07 18:54:09 [] ? child_rip+0x0/0x20 2014-01-07 18:54:09 kintegrityd/0 S 0000000000000000 0 170 2 0x00000000 2014-01-07 18:54:09 ffff880833e57e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:09 0000000000000001 ffff88085c536740 ffff880833e57de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e55ab8 ffff880833e57fd8 000000000000fb88 ffff880833e55ab8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 0000000000000001 0 171 2 0x00000000 2014-01-07 18:54:10 ffff880833e5be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e5bde0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e55058 ffff880833e5bfd8 000000000000fb88 ffff880833e55058 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/2 S 0000000000000002 0 172 2 0x00000000 2014-01-07 18:54:10 ffff880833e5de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e5dde0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e545f8 ffff880833e5dfd8 000000000000fb88 ffff880833e545f8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/3 S 0000000000000003 0 173 2 0x00000000 2014-01-07 18:54:10 ffff880833e61e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e61de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e5faf8 ffff880833e61fd8 000000000000fb88 ffff880833e5faf8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/4 S 0000000000000004 0 174 2 0x00000000 2014-01-07 18:54:10 ffff880833e67e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e67de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e5f098 ffff880833e67fd8 000000000000fb88 ffff880833e5f098 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/5 S 0000000000000005 0 175 2 0x00000000 2014-01-07 18:54:10 ffff880833e69e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e69de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e5e638 ffff880833e69fd8 000000000000fb88 ffff880833e5e638 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/6 S 0000000000000006 0 176 2 0x00000000 2014-01-07 18:54:10 ffff880833e6de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e6dde0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e6bab8 ffff880833e6dfd8 000000000000fb88 ffff880833e6bab8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/7 S 0000000000000007 0 177 2 0x00000000 2014-01-07 18:54:10 ffff880833e71e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e71de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e6b058 ffff880833e71fd8 000000000000fb88 ffff880833e6b058 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/8 S 0000000000000008 0 178 2 0x00000000 2014-01-07 18:54:10 ffff880833e73e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e73de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e6a5f8 ffff880833e73fd8 000000000000fb88 ffff880833e6a5f8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/9 S 0000000000000009 0 179 2 0x00000000 2014-01-07 18:54:10 ffff880833e77e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e77de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e75af8 ffff880833e77fd8 000000000000fb88 ffff880833e75af8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 000000000000000a 0 180 2 0x00000000 2014-01-07 18:54:10 ffff880833e7be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e7bde0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e75098 ffff880833e7bfd8 000000000000fb88 ffff880833e75098 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 000000000000000b 0 181 2 0x00000000 2014-01-07 18:54:10 ffff880833e7fe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e7fde0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e74638 ffff880833e7ffd8 000000000000fb88 ffff880833e74638 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 000000000000000c 0 182 2 0x00000000 2014-01-07 18:54:10 ffff880833e83e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e83de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e81ab8 ffff880833e83fd8 000000000000fb88 ffff880833e81ab8 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 000000000000000d 0 183 2 0x00000000 2014-01-07 18:54:10 ffff880833e87e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:10 0000000000000001 ffff88085c536740 ffff880833e87de0 ffffffff8106669b 2014-01-07 18:54:10 ffff880833e81058 ffff880833e87fd8 000000000000fb88 ffff880833e81058 2014-01-07 18:54:10 Call Trace: 2014-01-07 18:54:10 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:10 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:10 [] kthread+0x96/0xa0 2014-01-07 18:54:10 [] child_rip+0xa/0x20 2014-01-07 18:54:10 [] ? kthread+0x0/0xa0 2014-01-07 18:54:10 [] ? child_rip+0x0/0x20 2014-01-07 18:54:10 kintegrityd/1 S 000000000000000e 0 184 2 0x00000000 2014-01-07 18:54:10 ffff880833e89e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:11 0000000000000001 ffff88085c536740 ffff880833e89de0 ffffffff8106669b 2014-01-07 18:54:11 ffff880833e805f8 ffff880833e89fd8 000000000000fb88 ffff880833e805f8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kintegrityd/1 S 000000000000000f 0 185 2 0x00000000 2014-01-07 18:54:11 ffff880833e8de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:11 0000000000000001 ffff88085c536740 ffff880833e8dde0 ffffffff8106669b 2014-01-07 18:54:11 ffff880833e8baf8 ffff880833e8dfd8 000000000000fb88 ffff880833e8baf8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/0 S 0000000000000000 0 202 2 0x00000000 2014-01-07 18:54:11 ffff880833f8de30 0000000000000046 0000000000000000 ffff880833f8ddf4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833ec05f8 ffff880833f8dfd8 000000000000fb88 ffff880833ec05f8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/1 S 0000000000000001 0 203 2 0x00000000 2014-01-07 18:54:11 ffff880833f91e30 0000000000000046 0000000000000000 ffff880044639908 2014-01-07 18:54:11 ffff880833f91db0 ffffffff8125bd22 ffff880833f91dd0 00000000bffe4c40 2014-01-07 18:54:11 ffff880833f8faf8 ffff880833f91fd8 000000000000fb88 ffff880833f8faf8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? __generic_unplug_device+0x32/0x40 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/2 S 0000000000000002 0 204 2 0x00000000 2014-01-07 18:54:11 ffff880833f95e30 0000000000000046 0000000000000000 ffff880833f95df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833f8f098 ffff880833f95fd8 000000000000fb88 ffff880833f8f098 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/3 S 0000000000000003 0 205 2 0x00000000 2014-01-07 18:54:11 ffff880833f97e30 0000000000000046 0000000000000000 ffff880833f97df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833f8e638 ffff880833f97fd8 000000000000fb88 ffff880833f8e638 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/4 S 0000000000000004 0 206 2 0x00000000 2014-01-07 18:54:11 ffff880833f9be30 0000000000000046 0000000000000000 ffff880833f9bdf4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833f99ab8 ffff880833f9bfd8 000000000000fb88 ffff880833f99ab8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/5 S 0000000000000005 0 207 2 0x00000000 2014-01-07 18:54:11 ffff880833f9fe30 0000000000000046 0000000000000000 ffff880833f9fdf4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833f99058 ffff880833f9ffd8 000000000000fb88 ffff880833f99058 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/6 S 0000000000000006 0 208 2 0x00000000 2014-01-07 18:54:11 ffff880833fa1e30 0000000000000046 0000000000000000 ffff880833fa1df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833f985f8 ffff880833fa1fd8 000000000000fb88 ffff880833f985f8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/7 S 0000000000000007 0 209 2 0x00000000 2014-01-07 18:54:11 ffff880833fa7e30 0000000000000046 0000000000000000 ffff880833fa7df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:54:11 ffff880833fa5af8 ffff880833fa7fd8 000000000000fb88 ffff880833fa5af8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/8 S 0000000000000008 0 210 2 0x00000000 2014-01-07 18:54:11 ffff880833fabe30 0000000000000046 0000000000000000 ffff880833fabdf4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:11 ffff880833fa5098 ffff880833fabfd8 000000000000fb88 ffff880833fa5098 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/9 S 0000000000000009 0 211 2 0x00000000 2014-01-07 18:54:11 ffff880833fade30 0000000000000046 ffff880833faddf8 ffff880833faddf4 2014-01-07 18:54:11 ffff880833faddb0 ffff88083fe82200 ffff880044636740 0000000000000207 2014-01-07 18:54:11 ffff880833fa4638 ffff880833fadfd8 000000000000fb88 ffff880833fa4638 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/10 S 000000000000000a 0 212 2 0x00000000 2014-01-07 18:54:11 ffff880833fb1e30 0000000000000046 0000000000000000 ffff880833fb1df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:54:11 ffff880833fafab8 ffff880833fb1fd8 000000000000fb88 ffff880833fafab8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/11 S 000000000000000b 0 213 2 0x00000000 2014-01-07 18:54:11 ffff880833fb5e30 0000000000000046 0000000000000000 ffff880833fb5df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe82600 ffff880044636740 00000000000001ff 2014-01-07 18:54:11 ffff880833faf058 ffff880833fb5fd8 000000000000fb88 ffff880833faf058 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:11 [] ? kthread+0x0/0xa0 2014-01-07 18:54:11 [] ? child_rip+0x0/0x20 2014-01-07 18:54:11 kblockd/12 S 000000000000000c 0 214 2 0x00000000 2014-01-07 18:54:11 ffff880833fb7e30 0000000000000046 0000000000000000 ffff880833fb7df4 2014-01-07 18:54:11 ffff880800000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:54:11 ffff880833fae5f8 ffff880833fb7fd8 000000000000fb88 ffff880833fae5f8 2014-01-07 18:54:11 Call Trace: 2014-01-07 18:54:11 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:11 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:11 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:11 [] kthread+0x96/0xa0 2014-01-07 18:54:11 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kblockd/13 S 000000000000000d 0 215 2 0x00000000 2014-01-07 18:54:12 ffff880833fbbe30 0000000000000046 0000000000000000 ffff880833fbbdf4 2014-01-07 18:54:12 ffff880800000000 ffff88083fe82a00 ffff880044636740 00000000000001ff 2014-01-07 18:54:12 ffff880833fb9af8 ffff880833fbbfd8 000000000000fb88 ffff880833fb9af8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kblockd/14 S 000000000000000e 0 216 2 0x00000000 2014-01-07 18:54:12 ffff880833fc1e30 0000000000000046 ffff880833fc1df8 ffff880833fc1df4 2014-01-07 18:54:12 ffff880800000000 ffff88083fe82c00 ffff880044636740 00000000000001ff 2014-01-07 18:54:12 ffff880833fb9098 ffff880833fc1fd8 000000000000fb88 ffff880833fb9098 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kblockd/15 S 000000000000000f 0 217 2 0x00000000 2014-01-07 18:54:12 ffff880833fc3e30 0000000000000046 ffff880833fc3df8 ffff880833fc3df4 2014-01-07 18:54:12 ffff880800000000 ffff88083fe82e00 ffff880044636740 00000000000001ff 2014-01-07 18:54:12 ffff880833fb8638 ffff880833fc3fd8 000000000000fb88 ffff880833fb8638 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? blk_unplug_work+0x0/0x70 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kacpid S 0000000000000000 0 234 2 0x00000000 2014-01-07 18:54:12 ffff880833821e30 0000000000000046 ffff8808344778f8 ffff880833f87b60 2014-01-07 18:54:12 0000000000000000 ffff880833b7a780 ffff880833821e10 0000000000000282 2014-01-07 18:54:12 ffff880833ffb098 ffff880833821fd8 000000000000fb88 ffff880833ffb098 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:12 [] ? acpi_os_execute_deferred+0x0/0x36 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kacpi_notify S 0000000000000000 0 235 2 0x00000000 2014-01-07 18:54:12 ffff880833823e30 0000000000000046 ffff880833823db0 ffffffff812f7c4e 2014-01-07 18:54:12 ffff8810343b6df8 0000000000000010 ffff880833823e10 0000000000000282 2014-01-07 18:54:12 ffff880833ffa638 ffff880833823fd8 000000000000fb88 ffff880833ffa638 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? acpi_hw_write_port+0x43/0x98 2014-01-07 18:54:12 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:12 [] ? acpi_os_execute_deferred+0x0/0x36 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 kacpi_hotplug S 0000000000000000 0 236 2 0x00000000 2014-01-07 18:54:12 ffff880833827e30 0000000000000046 0000000000000000 ffffffff8150f17d 2014-01-07 18:54:12 ffff880833827e10 ffffffff8106a703 ffff880833827e10 0000000000000286 2014-01-07 18:54:12 ffff880833825ab8 ffff880833827fd8 000000000000fb88 ffff880833825ab8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? wait_for_completion+0x1d/0x20 2014-01-07 18:54:12 [] ? set_cpus_allowed_ptr+0x133/0x1d0 2014-01-07 18:54:12 [] ? bind_to_cpu0+0x0/0x32 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/0 S 0000000000000000 0 237 2 0x00000000 2014-01-07 18:54:12 ffff8808334a9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334a9de0 ffffffff8106669b 2014-01-07 18:54:12 ffff880833825058 ffff8808334a9fd8 000000000000fb88 ffff880833825058 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/1 S 0000000000000001 0 238 2 0x00000000 2014-01-07 18:54:12 ffff8808334abe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334abde0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808338245f8 ffff8808334abfd8 000000000000fb88 ffff8808338245f8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/2 S 0000000000000002 0 239 2 0x00000000 2014-01-07 18:54:12 ffff8808334afe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334afde0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334adaf8 ffff8808334affd8 000000000000fb88 ffff8808334adaf8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/3 S 0000000000000003 0 240 2 0x00000000 2014-01-07 18:54:12 ffff8808334b3e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334b3de0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334ad098 ffff8808334b3fd8 000000000000fb88 ffff8808334ad098 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/4 S 0000000000000004 0 241 2 0x00000000 2014-01-07 18:54:12 ffff8808334b5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334b5de0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334ac638 ffff8808334b5fd8 000000000000fb88 ffff8808334ac638 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/5 S 0000000000000005 0 242 2 0x00000000 2014-01-07 18:54:12 ffff8808334b9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334b9de0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334b7ab8 ffff8808334b9fd8 000000000000fb88 ffff8808334b7ab8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/6 S 0000000000000006 0 243 2 0x00000000 2014-01-07 18:54:12 ffff8808334bde30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334bdde0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334b7058 ffff8808334bdfd8 000000000000fb88 ffff8808334b7058 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/7 S 0000000000000007 0 244 2 0x00000000 2014-01-07 18:54:12 ffff8808334c1e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334c1de0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334b65f8 ffff8808334c1fd8 000000000000fb88 ffff8808334b65f8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:12 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:12 [] kthread+0x96/0xa0 2014-01-07 18:54:12 [] child_rip+0xa/0x20 2014-01-07 18:54:12 [] ? kthread+0x0/0xa0 2014-01-07 18:54:12 [] ? child_rip+0x0/0x20 2014-01-07 18:54:12 ata/8 S 0000000000000008 0 245 2 0x00000000 2014-01-07 18:54:12 ffff8808334c5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:12 0000000000000001 ffff88085c536740 ffff8808334c5de0 ffffffff8106669b 2014-01-07 18:54:12 ffff8808334c3af8 ffff8808334c5fd8 000000000000fb88 ffff8808334c3af8 2014-01-07 18:54:12 Call Trace: 2014-01-07 18:54:12 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/9 S 0000000000000009 0 246 2 0x00000000 2014-01-07 18:54:13 ffff8808334c9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334c9de0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334c3098 ffff8808334c9fd8 000000000000fb88 ffff8808334c3098 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/10 S 000000000000000a 0 247 2 0x00000000 2014-01-07 18:54:13 ffff8808334cbe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334cbde0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334c2638 ffff8808334cbfd8 000000000000fb88 ffff8808334c2638 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/11 S 000000000000000b 0 248 2 0x00000000 2014-01-07 18:54:13 ffff8808334cfe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334cfde0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334cdab8 ffff8808334cffd8 000000000000fb88 ffff8808334cdab8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/12 S 000000000000000c 0 249 2 0x00000000 2014-01-07 18:54:13 ffff8808334d3e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334d3de0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334cd058 ffff8808334d3fd8 000000000000fb88 ffff8808334cd058 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/13 S 000000000000000d 0 250 2 0x00000000 2014-01-07 18:54:13 ffff8808334d5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334d5de0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334cc5f8 ffff8808334d5fd8 000000000000fb88 ffff8808334cc5f8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/14 S 000000000000000e 0 251 2 0x00000000 2014-01-07 18:54:13 ffff8808334dbe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334dbde0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334d9af8 ffff8808334dbfd8 000000000000fb88 ffff8808334d9af8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata/15 S 000000000000000f 0 252 2 0x00000000 2014-01-07 18:54:13 ffff8808334dfe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff8808334dfde0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808334d9098 ffff8808334dffd8 000000000000fb88 ffff8808334d9098 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ata_aux S 0000000000000019 0 269 2 0x00000000 2014-01-07 18:54:13 ffff88083351be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff88083351bde0 ffffffff8106669b 2014-01-07 18:54:13 ffff880833519af8 ffff88083351bfd8 000000000000fb88 ffff880833519af8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 ksuspend_usbd S 0000000000000019 0 270 2 0x00000000 2014-01-07 18:54:13 ffff88083351fe30 0000000000000046 0000000000000000 0000000000000402 2014-01-07 18:54:13 ffff88083351fdf0 ffffffff813bb41c ffff88083351fde0 ffffffff8106669b 2014-01-07 18:54:13 ffff880833519098 ffff88083351ffd8 000000000000fb88 ffff880833519098 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? usb_suspend_both+0x1bc/0x340 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:13 [] ? usb_autosuspend_work+0x0/0x20 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 khubd S 0000000000000000 0 271 2 0x00000000 2014-01-07 18:54:13 ffff880833521d60 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:13 ffff880831aa3800 00000400ffffffff ffff880800000000 ffff880831617c00 2014-01-07 18:54:13 ffff880833518638 ffff880833521fd8 000000000000fb88 ffff880833518638 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] hub_thread+0x379/0x1a00 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? hub_thread+0x0/0x1a00 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 kseriod S 0000000000000019 0 272 2 0x00000000 2014-01-07 18:54:13 ffff880833527e50 0000000000000046 0000000000000000 ffffffff8106669b 2014-01-07 18:54:13 ffff880833527e10 ffffffff810096f0 ffff880833527e30 0000000000000286 2014-01-07 18:54:13 ffff880833525ab8 ffff880833527fd8 000000000000fb88 ffff880833525ab8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? __switch_to+0xd0/0x320 2014-01-07 18:54:13 [] serio_thread+0x39a/0x450 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? serio_thread+0x0/0x450 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 md/0 S 0000000000000000 0 273 2 0x00000000 2014-01-07 18:54:13 ffff88083352be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff88083352bde0 ffffffff8106669b 2014-01-07 18:54:13 ffff880833525058 ffff88083352bfd8 000000000000fb88 ffff880833525058 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 md/1 S 0000000000000001 0 274 2 0x00000000 2014-01-07 18:54:13 ffff88083352de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:13 0000000000000001 ffff88085c536740 ffff88083352dde0 ffffffff8106669b 2014-01-07 18:54:13 ffff8808335245f8 ffff88083352dfd8 000000000000fb88 ffff8808335245f8 2014-01-07 18:54:13 Call Trace: 2014-01-07 18:54:13 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:13 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:13 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:13 [] kthread+0x96/0xa0 2014-01-07 18:54:13 [] child_rip+0xa/0x20 2014-01-07 18:54:13 [] ? kthread+0x0/0xa0 2014-01-07 18:54:13 [] ? child_rip+0x0/0x20 2014-01-07 18:54:13 md/2 S 0000000000000002 0 275 2 0x00000000 2014-01-07 18:54:13 ffff880833531e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833531de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083352faf8 ffff880833531fd8 000000000000fb88 ffff88083352faf8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/3 S 0000000000000003 0 276 2 0x00000000 2014-01-07 18:54:14 ffff880833535e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833535de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083352f098 ffff880833535fd8 000000000000fb88 ffff88083352f098 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/4 S 0000000000000004 0 277 2 0x00000000 2014-01-07 18:54:14 ffff880833537e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833537de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083352e638 ffff880833537fd8 000000000000fb88 ffff88083352e638 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/5 S 0000000000000005 0 278 2 0x00000000 2014-01-07 18:54:14 ffff88083353be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff88083353bde0 ffffffff8106669b 2014-01-07 18:54:14 ffff880833539ab8 ffff88083353bfd8 000000000000fb88 ffff880833539ab8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/6 S 0000000000000006 0 279 2 0x00000000 2014-01-07 18:54:14 ffff880833541e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833541de0 ffffffff8106669b 2014-01-07 18:54:14 ffff880833539058 ffff880833541fd8 000000000000fb88 ffff880833539058 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/7 S 0000000000000007 0 280 2 0x00000000 2014-01-07 18:54:14 ffff880833543e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833543de0 ffffffff8106669b 2014-01-07 18:54:14 ffff8808335385f8 ffff880833543fd8 000000000000fb88 ffff8808335385f8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/8 S 0000000000000008 0 281 2 0x00000000 2014-01-07 18:54:14 ffff880833547e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833547de0 ffffffff8106669b 2014-01-07 18:54:14 ffff880833545af8 ffff880833547fd8 000000000000fb88 ffff880833545af8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/9 S 0000000000000009 0 282 2 0x00000000 2014-01-07 18:54:14 ffff88083354be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff88083354bde0 ffffffff8106669b 2014-01-07 18:54:14 ffff880833545098 ffff88083354bfd8 000000000000fb88 ffff880833545098 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/10 S 000000000000000a 0 283 2 0x00000000 2014-01-07 18:54:14 ffff88083354de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff88083354dde0 ffffffff8106669b 2014-01-07 18:54:14 ffff880833544638 ffff88083354dfd8 000000000000fb88 ffff880833544638 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/11 S 000000000000000b 0 284 2 0x00000000 2014-01-07 18:54:14 ffff880833551e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833551de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083354fab8 ffff880833551fd8 000000000000fb88 ffff88083354fab8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/12 S 000000000000000c 0 285 2 0x00000000 2014-01-07 18:54:14 ffff880833555e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833555de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083354f058 ffff880833555fd8 000000000000fb88 ffff88083354f058 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/13 S 000000000000000d 0 286 2 0x00000000 2014-01-07 18:54:14 ffff880833559e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833559de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083354e5f8 ffff880833559fd8 000000000000fb88 ffff88083354e5f8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/14 S 000000000000000e 0 287 2 0x00000000 2014-01-07 18:54:14 ffff88083355de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff88083355dde0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083355baf8 ffff88083355dfd8 000000000000fb88 ffff88083355baf8 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:14 md/15 S 000000000000000f 0 288 2 0x00000000 2014-01-07 18:54:14 ffff880833561e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:14 0000000000000001 ffff88085c536740 ffff880833561de0 ffffffff8106669b 2014-01-07 18:54:14 ffff88083355b098 ffff880833561fd8 000000000000fb88 ffff88083355b098 2014-01-07 18:54:14 Call Trace: 2014-01-07 18:54:14 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:14 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:14 [] kthread+0x96/0xa0 2014-01-07 18:54:14 [] child_rip+0xa/0x20 2014-01-07 18:54:14 [] ? kthread+0x0/0xa0 2014-01-07 18:54:14 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/0 S 0000000000000000 0 305 2 0x00000000 2014-01-07 18:54:15 ffff88083359de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff88083359dde0 ffffffff8106669b 2014-01-07 18:54:15 ffff88083359baf8 ffff88083359dfd8 000000000000fb88 ffff88083359baf8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/1 S 0000000000000001 0 306 2 0x00000000 2014-01-07 18:54:15 ffff8808335a1e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335a1de0 ffffffff8106669b 2014-01-07 18:54:15 ffff88083359b098 ffff8808335a1fd8 000000000000fb88 ffff88083359b098 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/2 S 0000000000000002 0 307 2 0x00000000 2014-01-07 18:54:15 ffff8808335a5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335a5de0 ffffffff8106669b 2014-01-07 18:54:15 ffff88083359a638 ffff8808335a5fd8 000000000000fb88 ffff88083359a638 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/3 S 0000000000000003 0 308 2 0x00000000 2014-01-07 18:54:15 ffff8808335a9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335a9de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335a7ab8 ffff8808335a9fd8 000000000000fb88 ffff8808335a7ab8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/4 S 0000000000000004 0 309 2 0x00000000 2014-01-07 18:54:15 ffff8808335ade30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335adde0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335a7058 ffff8808335adfd8 000000000000fb88 ffff8808335a7058 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/5 S 0000000000000005 0 310 2 0x00000000 2014-01-07 18:54:15 ffff8808335afe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335afde0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335a65f8 ffff8808335affd8 000000000000fb88 ffff8808335a65f8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/6 S 0000000000000006 0 311 2 0x00000000 2014-01-07 18:54:15 ffff8808335b3e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335b3de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335b1af8 ffff8808335b3fd8 000000000000fb88 ffff8808335b1af8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/7 S 0000000000000007 0 312 2 0x00000000 2014-01-07 18:54:15 ffff8808335b7e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335b7de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335b1098 ffff8808335b7fd8 000000000000fb88 ffff8808335b1098 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/8 S 0000000000000008 0 313 2 0x00000000 2014-01-07 18:54:15 ffff8808335b9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335b9de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335b0638 ffff8808335b9fd8 000000000000fb88 ffff8808335b0638 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/9 S 0000000000000009 0 314 2 0x00000000 2014-01-07 18:54:15 ffff8808335bfe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335bfde0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335bdab8 ffff8808335bffd8 000000000000fb88 ffff8808335bdab8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/10 S 000000000000000a 0 315 2 0x00000000 2014-01-07 18:54:15 ffff8808335c3e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335c3de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335bd058 ffff8808335c3fd8 000000000000fb88 ffff8808335bd058 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/11 S 000000000000000b 0 316 2 0x00000000 2014-01-07 18:54:15 ffff8808335c5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335c5de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335bc5f8 ffff8808335c5fd8 000000000000fb88 ffff8808335bc5f8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/12 S 000000000000000c 0 317 2 0x00000000 2014-01-07 18:54:15 ffff8808335c9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335c9de0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335c7af8 ffff8808335c9fd8 000000000000fb88 ffff8808335c7af8 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/13 S 000000000000000d 0 318 2 0x00000000 2014-01-07 18:54:15 ffff8808335cde30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335cdde0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335c7098 ffff8808335cdfd8 000000000000fb88 ffff8808335c7098 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:15 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:15 [] kthread+0x96/0xa0 2014-01-07 18:54:15 [] child_rip+0xa/0x20 2014-01-07 18:54:15 [] ? kthread+0x0/0xa0 2014-01-07 18:54:15 [] ? child_rip+0x0/0x20 2014-01-07 18:54:15 md_misc/14 S 000000000000000e 0 319 2 0x00000000 2014-01-07 18:54:15 ffff8808335cfe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:15 0000000000000001 ffff88085c536740 ffff8808335cfde0 ffffffff8106669b 2014-01-07 18:54:15 ffff8808335c6638 ffff8808335cffd8 000000000000fb88 ffff8808335c6638 2014-01-07 18:54:15 Call Trace: 2014-01-07 18:54:15 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 md_misc/15 S 000000000000000f 0 320 2 0x00000000 2014-01-07 18:54:16 ffff8808335d3e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:16 0000000000000001 ffff88085c536740 ffff8808335d3de0 ffffffff8106669b 2014-01-07 18:54:16 ffff8808335d1ab8 ffff8808335d3fd8 000000000000fb88 ffff8808335d1ab8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 khungtaskd S 000000000000000c 0 337 2 0x00000000 2014-01-07 18:54:16 ffff880831fe1dd0 0000000000000046 0000000000000000 ffff880831fe1d94 2014-01-07 18:54:16 0000000000000000 0000000000000286 ffff880831fe1d70 ffffffff810810cc 2014-01-07 18:54:16 ffff880833608638 ffff880831fe1fd8 000000000000fb88 ffff880833608638 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:16 [] schedule_timeout_interruptible+0x1e/0x20 2014-01-07 18:54:16 [] watchdog+0x52/0x220 2014-01-07 18:54:16 [] ? watchdog+0x0/0x220 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 kswapd0 S 0000000000000000 0 338 2 0x00000000 2014-01-07 18:54:16 ffff880831fe5e20 0000000000000046 0000000000000000 000000808150f36a 2014-01-07 18:54:16 ffff88085c536740 00000000000000e4 0000000000000002 ffffffff81c07400 2014-01-07 18:54:16 ffff880831fe3ab8 ffff880831fe5fd8 000000000000fb88 ffff880831fe3ab8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] kswapd+0x311/0x3c0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? kswapd+0x0/0x3c0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 kswapd1 S 0000000000000008 0 339 2 0x00000000 2014-01-07 18:54:16 ffff880831fe9e20 0000000000000046 0000000000000000 000000808150f36a 2014-01-07 18:54:16 ffff880044736740 00000000000000fc 0000000000000002 ffffffff81c07400 2014-01-07 18:54:16 ffff880831fe3058 ffff880831fe9fd8 000000000000fb88 ffff880831fe3058 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] kswapd+0x311/0x3c0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? kswapd+0x0/0x3c0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 ksmd S 0000000000000019 0 340 2 0x00000000 2014-01-07 18:54:16 ffff88083180be20 0000000000000046 0000000000000000 ffff880831fe2040 2014-01-07 18:54:16 ffff88083180bdb0 ffffffff81065deb ffff88083180bdb0 ffff880831fe2040 2014-01-07 18:54:16 ffff880831fe25f8 ffff88083180bfd8 000000000000fb88 ffff880831fe25f8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? enqueue_task_fair+0xfb/0x100 2014-01-07 18:54:16 [] ksm_scan_thread+0x81b/0xd80 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? ksm_scan_thread+0x0/0xd80 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 khugepaged S 0000000000000000 0 341 2 0x00000000 2014-01-07 18:54:16 ffff88083180fce0 0000000000000046 0000000000000000 ffff88083180fca4 2014-01-07 18:54:16 ffff880800000000 0000000000000282 ffff88083180fc80 ffffffff810810cc 2014-01-07 18:54:16 ffff88083180daf8 ffff88083180ffd8 000000000000fb88 ffff88083180daf8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:16 [] khugepaged+0x4ec/0x1310 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? khugepaged+0x0/0x1310 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/0 S 0000000000000000 0 342 2 0x00000000 2014-01-07 18:54:16 ffff880831883e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff88083180d098 ffff880831883fd8 000000000000fb88 ffff88083180d098 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/1 S 0000000000000001 0 343 2 0x00000000 2014-01-07 18:54:16 ffff880831885e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff88083180c638 ffff880831885fd8 000000000000fb88 ffff88083180c638 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/2 S 0000000000000002 0 344 2 0x00000000 2014-01-07 18:54:16 ffff880831889e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff880831887ab8 ffff880831889fd8 000000000000fb88 ffff880831887ab8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/3 S 0000000000000003 0 345 2 0x00000000 2014-01-07 18:54:16 ffff88083188de30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff880831887058 ffff88083188dfd8 000000000000fb88 ffff880831887058 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/4 S 0000000000000004 0 346 2 0x00000000 2014-01-07 18:54:16 ffff88083188fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff8808318865f8 ffff88083188ffd8 000000000000fb88 ffff8808318865f8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/5 S 0000000000000005 0 347 2 0x00000000 2014-01-07 18:54:16 ffff8808318d3e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff8808318d1af8 ffff8808318d3fd8 000000000000fb88 ffff8808318d1af8 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/6 S 0000000000000006 0 348 2 0x00000000 2014-01-07 18:54:16 ffff8808318d7e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff8808318d1098 ffff8808318d7fd8 000000000000fb88 ffff8808318d1098 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/7 S 0000000000000007 0 349 2 0x00000000 2014-01-07 18:54:16 ffff8808318dbe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:16 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:16 ffff8808318d0638 ffff8808318dbfd8 000000000000fb88 ffff8808318d0638 2014-01-07 18:54:16 Call Trace: 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:16 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:16 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:16 [] kthread+0x96/0xa0 2014-01-07 18:54:16 [] child_rip+0xa/0x20 2014-01-07 18:54:16 [] ? kthread+0x0/0xa0 2014-01-07 18:54:16 [] ? child_rip+0x0/0x20 2014-01-07 18:54:16 aio/8 S 0000000000000008 0 350 2 0x00000000 2014-01-07 18:54:16 ffff8808318dfe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318ddab8 ffff8808318dffd8 000000000000fb88 ffff8808318ddab8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/9 S 0000000000000009 0 351 2 0x00000000 2014-01-07 18:54:17 ffff8808318e3e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318dd058 ffff8808318e3fd8 000000000000fb88 ffff8808318dd058 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/10 S 000000000000000a 0 352 2 0x00000000 2014-01-07 18:54:17 ffff8808318e5e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318dc5f8 ffff8808318e5fd8 000000000000fb88 ffff8808318dc5f8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/11 S 000000000000000b 0 353 2 0x00000000 2014-01-07 18:54:17 ffff8808318e9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318e7af8 ffff8808318e9fd8 000000000000fb88 ffff8808318e7af8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/12 S 000000000000000c 0 354 2 0x00000000 2014-01-07 18:54:17 ffff8808318ede30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318e7098 ffff8808318edfd8 000000000000fb88 ffff8808318e7098 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/13 S 000000000000000d 0 355 2 0x00000000 2014-01-07 18:54:17 ffff8808318efe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318e6638 ffff8808318effd8 000000000000fb88 ffff8808318e6638 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/14 S 000000000000000e 0 356 2 0x00000000 2014-01-07 18:54:17 ffff8808318f5e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318f3ab8 ffff8808318f5fd8 000000000000fb88 ffff8808318f3ab8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 aio/15 S 000000000000000f 0 357 2 0x00000000 2014-01-07 18:54:17 ffff8808318f9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808318f3058 ffff8808318f9fd8 000000000000fb88 ffff8808318f3058 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/0 S 0000000000000000 0 374 2 0x00000000 2014-01-07 18:54:17 ffff880831973e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff880831971ab8 ffff880831973fd8 000000000000fb88 ffff880831971ab8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/1 S 0000000000000001 0 375 2 0x00000000 2014-01-07 18:54:17 ffff880831977e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff880831971058 ffff880831977fd8 000000000000fb88 ffff880831971058 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/2 S 0000000000000002 0 376 2 0x00000000 2014-01-07 18:54:17 ffff880831979e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff8808319705f8 ffff880831979fd8 000000000000fb88 ffff8808319705f8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/3 S 0000000000000003 0 377 2 0x00000000 2014-01-07 18:54:17 ffff88083197fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff88083197daf8 ffff88083197ffd8 000000000000fb88 ffff88083197daf8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/4 S 0000000000000004 0 378 2 0x00000000 2014-01-07 18:54:17 ffff880831983e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff88083197d098 ffff880831983fd8 000000000000fb88 ffff88083197d098 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/5 S 0000000000000005 0 379 2 0x00000000 2014-01-07 18:54:17 ffff880831985e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff88083197c638 ffff880831985fd8 000000000000fb88 ffff88083197c638 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/6 S 0000000000000006 0 380 2 0x00000000 2014-01-07 18:54:17 ffff880831989e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:17 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:17 ffff880831987ab8 ffff880831989fd8 000000000000fb88 ffff880831987ab8 2014-01-07 18:54:17 Call Trace: 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:17 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:17 [] kthread+0x96/0xa0 2014-01-07 18:54:17 [] child_rip+0xa/0x20 2014-01-07 18:54:17 [] ? kthread+0x0/0xa0 2014-01-07 18:54:17 [] ? child_rip+0x0/0x20 2014-01-07 18:54:17 crypto/7 S 0000000000000007 0 381 2 0x00000000 2014-01-07 18:54:17 ffff88083198de30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff880831987058 ffff88083198dfd8 000000000000fb88 ffff880831987058 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/8 S 0000000000000008 0 382 2 0x00000000 2014-01-07 18:54:18 ffff88083198fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff8808319865f8 ffff88083198ffd8 000000000000fb88 ffff8808319865f8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/9 S 0000000000000009 0 383 2 0x00000000 2014-01-07 18:54:18 ffff880831993e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff880831991af8 ffff880831993fd8 000000000000fb88 ffff880831991af8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/10 S 000000000000000a 0 384 2 0x00000000 2014-01-07 18:54:18 ffff880831999e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff880831991098 ffff880831999fd8 000000000000fb88 ffff880831991098 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/11 S 000000000000000b 0 385 2 0x00000000 2014-01-07 18:54:18 ffff88083199be30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff880831990638 ffff88083199bfd8 000000000000fb88 ffff880831990638 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/12 S 000000000000000c 0 386 2 0x00000000 2014-01-07 18:54:18 ffff88083199fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff88083199dab8 ffff88083199ffd8 000000000000fb88 ffff88083199dab8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/13 S 000000000000000d 0 387 2 0x00000000 2014-01-07 18:54:18 ffff8808319a3e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff88083199d058 ffff8808319a3fd8 000000000000fb88 ffff88083199d058 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/14 S 000000000000000e 0 388 2 0x00000000 2014-01-07 18:54:18 ffff8808319a5e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff88083199c5f8 ffff8808319a5fd8 000000000000fb88 ffff88083199c5f8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 crypto/15 S 000000000000000f 0 389 2 0x00000000 2014-01-07 18:54:18 ffff8808319a9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:18 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:18 ffff8808319a7af8 ffff8808319a9fd8 000000000000fb88 ffff8808319a7af8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/0 S 0000000000000000 0 410 2 0x00000000 2014-01-07 18:54:18 ffff8808319f5e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff8808319f5de0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319f3ab8 ffff8808319f5fd8 000000000000fb88 ffff8808319f3ab8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/1 S 0000000000000001 0 411 2 0x00000000 2014-01-07 18:54:18 ffff8808319f9e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff8808319f9de0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319f3058 ffff8808319f9fd8 000000000000fb88 ffff8808319f3058 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/2 S 0000000000000002 0 412 2 0x00000000 2014-01-07 18:54:18 ffff8808319fde30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff8808319fdde0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319f25f8 ffff8808319fdfd8 000000000000fb88 ffff8808319f25f8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/3 S 0000000000000003 0 413 2 0x00000000 2014-01-07 18:54:18 ffff880831a11e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff880831a11de0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319ffaf8 ffff880831a11fd8 000000000000fb88 ffff8808319ffaf8 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/4 S 0000000000000004 0 414 2 0x00000000 2014-01-07 18:54:18 ffff880831a15e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff880831a15de0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319ff098 ffff880831a15fd8 000000000000fb88 ffff8808319ff098 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] kthread+0x96/0xa0 2014-01-07 18:54:18 [] child_rip+0xa/0x20 2014-01-07 18:54:18 [] ? kthread+0x0/0xa0 2014-01-07 18:54:18 [] ? child_rip+0x0/0x20 2014-01-07 18:54:18 kthrotld/5 S 0000000000000005 0 415 2 0x00000000 2014-01-07 18:54:18 ffff880831a17e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:18 0000000000000001 ffff88085c536740 ffff880831a17de0 ffffffff8106669b 2014-01-07 18:54:18 ffff8808319fe638 ffff880831a17fd8 000000000000fb88 ffff8808319fe638 2014-01-07 18:54:18 Call Trace: 2014-01-07 18:54:18 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:18 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:18 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/6 S 0000000000000006 0 416 2 0x00000000 2014-01-07 18:54:19 ffff880831a1be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a1bde0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a19ab8 ffff880831a1bfd8 000000000000fb88 ffff880831a19ab8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/7 S 0000000000000007 0 417 2 0x00000000 2014-01-07 18:54:19 ffff880831a1fe30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a1fde0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a19058 ffff880831a1ffd8 000000000000fb88 ffff880831a19058 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/8 S 0000000000000008 0 418 2 0x00000000 2014-01-07 18:54:19 ffff880831a21e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a21de0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a185f8 ffff880831a21fd8 000000000000fb88 ffff880831a185f8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/9 S 0000000000000009 0 419 2 0x00000000 2014-01-07 18:54:19 ffff880831a27e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a27de0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a25af8 ffff880831a27fd8 000000000000fb88 ffff880831a25af8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/10 S 000000000000000a 0 420 2 0x00000000 2014-01-07 18:54:19 ffff880831a2be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a2bde0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a25098 ffff880831a2bfd8 000000000000fb88 ffff880831a25098 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/11 S 000000000000000b 0 421 2 0x00000000 2014-01-07 18:54:19 ffff880831a2de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a2dde0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a24638 ffff880831a2dfd8 000000000000fb88 ffff880831a24638 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/12 S 000000000000000c 0 422 2 0x00000000 2014-01-07 18:54:19 ffff880831a31e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a31de0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a2fab8 ffff880831a31fd8 000000000000fb88 ffff880831a2fab8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/13 S 000000000000000d 0 423 2 0x00000000 2014-01-07 18:54:19 ffff880831a35e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a35de0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a2f058 ffff880831a35fd8 000000000000fb88 ffff880831a2f058 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/14 S 000000000000000e 0 424 2 0x00000000 2014-01-07 18:54:19 ffff880831a37e30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a37de0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a2e5f8 ffff880831a37fd8 000000000000fb88 ffff880831a2e5f8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kthrotld/15 S 000000000000000f 0 425 2 0x00000000 2014-01-07 18:54:19 ffff880831a3be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 0000000000000001 ffff88085c536740 ffff880831a3bde0 ffffffff8106669b 2014-01-07 18:54:19 ffff880831a39af8 ffff880831a3bfd8 000000000000fb88 ffff880831a39af8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kipmi0 S 0000000000000008 0 449 2 0x00000000 2014-01-07 18:54:19 ffff880831abddb0 0000000000000046 0000000000000000 ffff880831abdd74 2014-01-07 18:54:19 ffff880800000000 0000000000000286 ffff880831abdd50 ffffffff810810cc 2014-01-07 18:54:19 ffff880831abf058 ffff880831abdfd8 000000000000fb88 ffff880831abf058 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:19 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:19 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:19 [] schedule_timeout_interruptible+0x1e/0x20 2014-01-07 18:54:19 [] ipmi_thread+0x1ba/0x1c0 2014-01-07 18:54:19 [] ? ipmi_thread+0x0/0x1c0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 kpsmoused S 0000000000000019 0 453 2 0x00000000 2014-01-07 18:54:19 ffff88083171be30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:19 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:19 ffff880833c445f8 ffff88083171bfd8 000000000000fb88 ffff880833c445f8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 usbhid_resume S 0000000000000019 0 454 2 0x00000000 2014-01-07 18:54:19 ffff88083176de30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:19 ffff88083176ddb0 ffffffff81057310 ffff880833c44ad8 ffff88085c5367a8 2014-01-07 18:54:19 ffff880833c45058 ffff88083176dfd8 000000000000fb88 ffff880833c45058 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 mlx4 S 0000000000000006 0 673 2 0x00000000 2014-01-07 18:54:19 ffff88083128fe30 0000000000000046 ffff880833d84728 ffffe8f7e7801ac8 2014-01-07 18:54:19 ffff88083128fda0 ffffffff81082228 ffff88083128fde0 ffffffff81091b89 2014-01-07 18:54:19 ffff88083128daf8 ffff88083128ffd8 000000000000fb88 ffff88083128daf8 2014-01-07 18:54:19 Call Trace: 2014-01-07 18:54:19 [] ? add_timer+0x18/0x30 2014-01-07 18:54:19 [] ? queue_delayed_work_on+0xb9/0x120 2014-01-07 18:54:19 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:19 [] ? mlx4_sense_port+0x0/0xd0 [mlx4_core] 2014-01-07 18:54:19 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:19 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:19 [] kthread+0x96/0xa0 2014-01-07 18:54:19 [] child_rip+0xa/0x20 2014-01-07 18:54:19 [] ? kthread+0x0/0xa0 2014-01-07 18:54:19 [] ? child_rip+0x0/0x20 2014-01-07 18:54:19 mlx4_en S 0000000000000019 0 681 2 0x00000000 2014-01-07 18:54:20 ffff88083113fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:20 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:20 ffff88083108a638 ffff88083113ffd8 000000000000fb88 ffff88083108a638 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 mlx4_en S 0000000000000019 0 682 2 0x00000000 2014-01-07 18:54:20 ffff88082d193e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:20 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:20 ffff8808310385f8 ffff88082d193fd8 000000000000fb88 ffff8808310385f8 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/0 S 0000000000000000 0 1215 2 0x00000000 2014-01-07 18:54:20 ffff88082784be30 0000000000000046 ffff88082784bda0 0000000000000246 2014-01-07 18:54:20 ffff88082784bda0 ffffffff8151103b ffff88082784be10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff8808310c05f8 ffff88082784bfd8 000000000000fb88 ffff8808310c05f8 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/1 S 0000000000000001 0 1216 2 0x00000000 2014-01-07 18:54:20 ffff880828e6fe30 0000000000000046 ffff880828e6fda0 0000000000000246 2014-01-07 18:54:20 ffff880828e6fda0 ffffffff8151103b ffff880828e6fe10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff8808310c1058 ffff880828e6ffd8 000000000000fb88 ffff8808310c1058 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/2 S 0000000000000002 0 1217 2 0x00000000 2014-01-07 18:54:20 ffff880827839e30 0000000000000046 0000000000000000 0000000000000246 2014-01-07 18:54:20 ffff880827839da0 ffffffff8151103b ffff880827839e10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff880828320638 ffff880827839fd8 000000000000fb88 ffff880828320638 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/3 S 0000000000000003 0 1218 2 0x00000000 2014-01-07 18:54:20 ffff880827807e30 0000000000000046 0000000000000000 ffffffff810914cf 2014-01-07 18:54:20 ffff880827807da0 ffffffffa00c907b ffff880827807e10 ffffffffa00c9acf 2014-01-07 18:54:20 ffff880828321098 ffff880827807fd8 000000000000fb88 ffff880828321098 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? queue_work+0x1f/0x30 2014-01-07 18:54:20 [] ? rpc_final_put_task+0x4b/0x60 [sunrpc] 2014-01-07 18:54:20 [] ? __rpc_execute+0x19f/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/4 S 0000000000000004 0 1219 2 0x00000000 2014-01-07 18:54:20 ffff880828307e30 0000000000000046 ffff880828307da0 0000000000000246 2014-01-07 18:54:20 ffff880828307da0 ffffffff8151103b ffff880828307dd0 ffffffffa00c9ea1 2014-01-07 18:54:20 ffff8808283aa638 ffff880828307fd8 000000000000fb88 ffff8808283aa638 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? rpc_wake_up_first+0xc1/0x1e0 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? xprt_autoclose+0x0/0x70 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/5 S 0000000000000005 0 1220 2 0x00000000 2014-01-07 18:54:20 ffff880831401e30 0000000000000046 ffff880831401da0 0000000000000246 2014-01-07 18:54:20 ffff880831401da0 ffffffff8151103b ffff880831401e10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff8808283ab098 ffff880831401fd8 000000000000fb88 ffff8808283ab098 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/6 S 0000000000000006 0 1221 2 0x00000000 2014-01-07 18:54:20 ffff880827863e30 0000000000000046 ffff880827863d90 ffffffff810914cf 2014-01-07 18:54:20 ffff880827863da0 ffffffffa00c907b ffff880827863e10 ffffffffa00c9acf 2014-01-07 18:54:20 ffff88082884fab8 ffff880827863fd8 000000000000fb88 ffff88082884fab8 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? queue_work+0x1f/0x30 2014-01-07 18:54:20 [] ? rpc_final_put_task+0x4b/0x60 [sunrpc] 2014-01-07 18:54:20 [] ? __rpc_execute+0x19f/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/7 S 0000000000000007 0 1222 2 0x00000000 2014-01-07 18:54:20 ffff88082782de30 0000000000000046 ffff88082782dd90 ffffffff810914cf 2014-01-07 18:54:20 ffff88082782dda0 ffffffffa00c907b ffff88082782de10 ffffffffa00c9acf 2014-01-07 18:54:20 ffff88082884e5f8 ffff88082782dfd8 000000000000fb88 ffff88082884e5f8 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? queue_work+0x1f/0x30 2014-01-07 18:54:20 [] ? rpc_final_put_task+0x4b/0x60 [sunrpc] 2014-01-07 18:54:20 [] ? __rpc_execute+0x19f/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/8 S 0000000000000008 0 1223 2 0x00000000 2014-01-07 18:54:20 ffff880828e7de30 0000000000000046 ffff880828e7dda0 0000000000000246 2014-01-07 18:54:20 ffff880828e7dda0 ffffffff8151103b ffff880828e7de10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff880828f245f8 ffff880828e7dfd8 000000000000fb88 ffff880828f245f8 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/9 S 0000000000000009 0 1224 2 0x00000000 2014-01-07 18:54:20 ffff880827829e30 0000000000000046 0000000000000000 0000000000000246 2014-01-07 18:54:20 ffff880827829da0 ffffffff8151103b ffff880827829e10 ffffffffa00c99f6 2014-01-07 18:54:20 ffff880831039058 ffff880827829fd8 000000000000fb88 ffff880831039058 2014-01-07 18:54:20 Call Trace: 2014-01-07 18:54:20 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:20 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:20 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:20 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:20 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:20 [] kthread+0x96/0xa0 2014-01-07 18:54:20 [] child_rip+0xa/0x20 2014-01-07 18:54:20 [] ? kthread+0x0/0xa0 2014-01-07 18:54:20 [] ? child_rip+0x0/0x20 2014-01-07 18:54:20 rpciod/10 S 000000000000000a 0 1225 2 0x00000000 2014-01-07 18:54:20 ffff880828e1be30 0000000000000046 0000000000000000 ffffe8ffff641c48 2014-01-07 18:54:20 ffff880828e1bdd0 ffff88082704e800 ffff88082704edd0 00000000ffffff95 2014-01-07 18:54:20 ffff880831070638 ffff880828e1bfd8 000000000000fb88 ffff880831070638 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? xs_tcp_setup_socket+0x0/0x4c0 [sunrpc] 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 rpciod/11 S 000000000000000b 0 1226 2 0x00000000 2014-01-07 18:54:21 ffff8808283efe30 0000000000000046 ffff8808283efda0 0000000000000246 2014-01-07 18:54:21 ffff8808283efda0 ffffffff8151103b ffff8808283efe10 ffffffffa00c99f6 2014-01-07 18:54:21 ffff880831acc638 ffff8808283effd8 000000000000fb88 ffff880831acc638 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:21 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:21 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:21 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 rpciod/12 S 000000000000000c 0 1227 2 0x00000000 2014-01-07 18:54:21 ffff880831065e30 0000000000000046 0000000000000000 0000000000000246 2014-01-07 18:54:21 ffff880831065da0 ffffffff8151103b ffff880831065e10 ffffffffa00c99f6 2014-01-07 18:54:21 ffff880831074638 ffff880831065fd8 000000000000fb88 ffff880831074638 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:21 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:21 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 rpciod/13 S 000000000000000d 0 1228 2 0x00000000 2014-01-07 18:54:21 ffff88082780be30 0000000000000046 0000000000000000 0000000000000246 2014-01-07 18:54:21 ffff88082780bda0 ffffffff8151103b ffff88082780be10 ffffffffa00c99f6 2014-01-07 18:54:21 ffff880831075af8 ffff88082780bfd8 000000000000fb88 ffff880831075af8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:21 [] ? __rpc_execute+0xc6/0x350 [sunrpc] 2014-01-07 18:54:21 [] ? rpc_async_schedule+0x0/0x40 [sunrpc] 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 rpciod/14 S 000000000000000e 0 1229 2 0x00000000 2014-01-07 18:54:21 ffff8808283f9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:21 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:21 ffff880831075098 ffff8808283f9fd8 000000000000fb88 ffff880831075098 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 rpciod/15 S 000000000000000f 0 1230 2 0x00000000 2014-01-07 18:54:21 ffff88082783de30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:21 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:21 ffff880831253af8 ffff88082783dfd8 000000000000fb88 ffff880831253af8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 kslowd000 S 0000000000000019 0 1248 2 0x00000000 2014-01-07 18:54:21 ffff88082836fe60 0000000000000046 0000000000000000 0000000a00000003 2014-01-07 18:54:21 ffffffffffffffff ffffffff81065deb ffff88082836fdf0 ffff88082836fe80 2014-01-07 18:54:21 ffff880828f4a5f8 ffff88082836ffd8 000000000000fb88 ffff880828f4a5f8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? enqueue_task_fair+0xfb/0x100 2014-01-07 18:54:21 [] slow_work_thread+0x32b/0x360 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? slow_work_thread+0x0/0x360 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 kslowd001 S 0000000000000019 0 1249 2 0x00000000 2014-01-07 18:54:21 ffff880833681e60 0000000000000046 0000000000000000 0000000a00000003 2014-01-07 18:54:21 ffffffffffffffff ffffffff81065deb ffff880833681df0 ffff880833681e80 2014-01-07 18:54:21 ffff880827827ab8 ffff880833681fd8 000000000000fb88 ffff880827827ab8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? enqueue_task_fair+0xfb/0x100 2014-01-07 18:54:21 [] slow_work_thread+0x32b/0x360 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? slow_work_thread+0x0/0x360 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 nfsiod S 0000000000000001 0 1250 2 0x00000000 2014-01-07 18:54:21 ffff880827847e30 0000000000000046 0000000000000000 ffff880827846000 2014-01-07 18:54:21 ffff8800bd47ef40 0000000000000000 ffff8800bd47ef40 ffffffffa00c9110 2014-01-07 18:54:21 ffff880827827058 ffff880827847fd8 000000000000fb88 ffff880827827058 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? rpc_async_release+0x0/0x20 [sunrpc] 2014-01-07 18:54:21 [] ? rpc_async_release+0x0/0x20 [sunrpc] 2014-01-07 18:54:21 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 kauditd S 0000000000000019 0 1437 2 0x00000000 2014-01-07 18:54:21 ffff880828357e70 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:21 ffff880828357e20 ffffffff81064b00 ffff88085c5367a8 ffff88085c536740 2014-01-07 18:54:21 ffff88083127f058 ffff880828357fd8 000000000000fb88 ffff88083127f058 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? pick_next_task_fair+0xd0/0x130 2014-01-07 18:54:21 [] kauditd_thread+0x149/0x180 2014-01-07 18:54:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:21 [] ? kauditd_thread+0x0/0x180 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 udevd S 0000000000000000 0 2171 1 0x00000000 2014-01-07 18:54:21 ffff8808329aba58 0000000000000082 0000000000000000 ffffffff81063422 2014-01-07 18:54:21 ffff8808329ab9f8 ffffffff81197726 0000000000000000 ffff880834305540 2014-01-07 18:54:21 ffff880828321af8 ffff8808329abfd8 000000000000fb88 ffff880828321af8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:21 [] ? pollwake+0x56/0x60 2014-01-07 18:54:21 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:21 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:21 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:21 [] ? unix_dgram_poll+0x127/0x1c0 2014-01-07 18:54:21 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:21 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:21 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:21 [] ? pollwake+0x0/0x60 2014-01-07 18:54:21 [] ? pollwake+0x0/0x60 2014-01-07 18:54:21 [] ? pollwake+0x0/0x60 2014-01-07 18:54:21 [] ? pollwake+0x0/0x60 2014-01-07 18:54:21 [] ? pollwake+0x0/0x60 2014-01-07 18:54:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:21 [] ? generic_file_aio_write+0xba/0x100 2014-01-07 18:54:21 [] ? do_sync_write+0xfa/0x140 2014-01-07 18:54:21 [] ? sys_recvfrom+0xee/0x180 2014-01-07 18:54:21 [] ? vfs_write+0x132/0x1a0 2014-01-07 18:54:21 [] sys_poll+0x71/0x100 2014-01-07 18:54:21 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:21 scsi_eh_0 S 0000000000000000 0 2702 2 0x00000000 2014-01-07 18:54:21 ffff880832243e30 0000000000000046 ffff880832243d90 0000000000000018 2014-01-07 18:54:21 ffff880832243df0 ffffffff81393e68 ffff880832243df0 ffffffff8139436a 2014-01-07 18:54:21 ffff880831dabaf8 ffff880832243fd8 000000000000fb88 ffff880831dabaf8 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:21 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:21 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:21 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:21 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 scsi_eh_1 S 0000000000000009 0 2703 2 0x00000000 2014-01-07 18:54:21 ffff8808271dbe30 0000000000000046 000000000000000f 0000000000000282 2014-01-07 18:54:21 ffff8808271dbdf0 ffffffff81393e68 ffff8808271dbdf0 ffffffff8139436a 2014-01-07 18:54:21 ffff880832808638 ffff8808271dbfd8 000000000000fb88 ffff880832808638 2014-01-07 18:54:21 Call Trace: 2014-01-07 18:54:21 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:21 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:21 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:21 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:21 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:21 [] kthread+0x96/0xa0 2014-01-07 18:54:21 [] child_rip+0xa/0x20 2014-01-07 18:54:21 [] ? kthread+0x0/0xa0 2014-01-07 18:54:21 [] ? child_rip+0x0/0x20 2014-01-07 18:54:21 scsi_eh_2 S 0000000000000019 0 2704 2 0x00000000 2014-01-07 18:54:21 ffff8808322abe30 0000000000000046 000000000000000f 0000000000000282 2014-01-07 18:54:21 ffff8808322abdf0 ffffffff81393e68 ffff8808322abdf0 ffffffff8139436a 2014-01-07 18:54:21 ffff880832604638 ffff8808322abfd8 000000000000fb88 ffff880832604638 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:22 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:22 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:22 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:22 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 scsi_eh_3 S 0000000000000019 0 2705 2 0x00000000 2014-01-07 18:54:22 ffff8808322ade30 0000000000000046 000000000000000f 0000000000000282 2014-01-07 18:54:22 ffff8808322addf0 ffffffff81393e68 ffff8808322addf0 ffffffff8139436a 2014-01-07 18:54:22 ffff880832493ab8 ffff8808322adfd8 000000000000fb88 ffff880832493ab8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:22 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:22 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:22 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:22 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 scsi_eh_4 S 0000000000000019 0 2706 2 0x00000000 2014-01-07 18:54:22 ffff8808322afe30 0000000000000046 000000000000000f 0000000000000282 2014-01-07 18:54:22 ffff8808322afdf0 ffffffff81393e68 ffff8808322afdf0 ffffffff8139436a 2014-01-07 18:54:22 ffff880831da5058 ffff8808322affd8 000000000000fb88 ffff880831da5058 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:22 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:22 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:22 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:22 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 scsi_eh_5 S 0000000000000019 0 2707 2 0x00000000 2014-01-07 18:54:22 ffff880832363e30 0000000000000046 0000000000000000 0000000000000282 2014-01-07 18:54:22 ffff880832363df0 ffffffff81393e68 ffff880832363df0 ffffffff8139436a 2014-01-07 18:54:22 ffff880831dab098 ffff880832363fd8 000000000000fb88 ffff880831dab098 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? ata_scsi_port_error_handler+0x348/0x710 2014-01-07 18:54:22 [] ? ata_scsi_cmd_error_handler+0x13a/0x190 2014-01-07 18:54:22 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:22 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:22 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 edac-poller S 000000000000000a 0 2745 2 0x00000000 2014-01-07 18:54:22 ffff880832a29e30 0000000000000046 0000000000000000 ffffe8f7e780ab80 2014-01-07 18:54:22 00000000ffffffff ffff880832020168 ffff880832a29fd8 ffffe8f7e780ab88 2014-01-07 18:54:22 ffff88083242baf8 ffff880832a29fd8 000000000000fb88 ffff88083242baf8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:22 [] ? edac_mc_workq_function+0x0/0xb0 [edac_core] 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kvm-irqfd-cle S 0000000000000014 0 3031 2 0x00000000 2014-01-07 18:54:22 ffff880827219e30 0000000000000046 0000000000000000 ffff8800447967a8 2014-01-07 18:54:22 ffff880827219db0 ffffffff81057310 ffff880832af80b8 ffff8800447967a8 2014-01-07 18:54:22 ffff880832af8638 ffff880827219fd8 000000000000fb88 ffff880832af8638 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kstriped S 0000000000000019 0 3045 2 0x00000000 2014-01-07 18:54:22 ffff88083236be30 0000000000000046 0000000000000000 ffff88085c5367a8 2014-01-07 18:54:22 ffff88083236bdb0 ffffffff81057310 ffff880832f96ad8 ffff88085c5367a8 2014-01-07 18:54:22 ffff880832f97058 ffff88083236bfd8 000000000000fb88 ffff880832f97058 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/0 S 0000000000000000 0 3047 2 0x00000000 2014-01-07 18:54:22 ffff88083236fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff8808323d0638 ffff88083236ffd8 000000000000fb88 ffff8808323d0638 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/1 S 0000000000000001 0 3048 2 0x00000000 2014-01-07 18:54:22 ffff880832819e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff8808329d9058 ffff880832819fd8 000000000000fb88 ffff8808329d9058 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/2 S 0000000000000002 0 3049 2 0x00000000 2014-01-07 18:54:22 ffff8808329cfe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff88083274baf8 ffff8808329cffd8 000000000000fb88 ffff88083274baf8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/3 S 0000000000000003 0 3050 2 0x00000000 2014-01-07 18:54:22 ffff880828ec3e30 0000000000000046 ffff880828ec3df8 ffff880828ec3df4 2014-01-07 18:54:22 ffff880828ec3da0 ffff88083fe81600 ffff880044636740 0000000000000519 2014-01-07 18:54:22 ffff880831acd098 ffff880828ec3fd8 000000000000fb88 ffff880831acd098 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? process_queued_ios+0x0/0x290 [dm_multipath] 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/4 S 0000000000000004 0 3051 2 0x00000000 2014-01-07 18:54:22 ffff880831d43e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff88082721a5f8 ffff880831d43fd8 000000000000fb88 ffff88082721a5f8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/5 S 0000000000000005 0 3052 2 0x00000000 2014-01-07 18:54:22 ffff880832aede30 0000000000000046 0000000000000000 ffff880832aeddf4 2014-01-07 18:54:22 ffff880832aeddf0 ffff88083fe81a00 ffff880044616740 0000000000000519 2014-01-07 18:54:22 ffff880828d93ab8 ffff880832aedfd8 000000000000fb88 ffff880828d93ab8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? process_queued_ios+0x0/0x290 [dm_multipath] 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/6 S 0000000000000006 0 3053 2 0x00000000 2014-01-07 18:54:22 ffff880831e0be30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff88083239f098 ffff880831e0bfd8 000000000000fb88 ffff88083239f098 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:22 [] kthread+0x96/0xa0 2014-01-07 18:54:22 [] child_rip+0xa/0x20 2014-01-07 18:54:22 [] ? kthread+0x0/0xa0 2014-01-07 18:54:22 [] ? child_rip+0x0/0x20 2014-01-07 18:54:22 kmpathd/7 S 0000000000000007 0 3054 2 0x00000000 2014-01-07 18:54:22 ffff880831139e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:22 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:22 ffff880827185ab8 ffff880831139fd8 000000000000fb88 ffff880827185ab8 2014-01-07 18:54:22 Call Trace: 2014-01-07 18:54:22 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/8 S 0000000000000008 0 3055 2 0x00000000 2014-01-07 18:54:23 ffff880828fd7e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff88082833c638 ffff880828fd7fd8 000000000000fb88 ffff88082833c638 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/9 S 0000000000000009 0 3056 2 0x00000000 2014-01-07 18:54:23 ffff8808329f9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff8808329f9da0 ffffffff81063422 ffff8808329f9df0 ffffffff81051439 2014-01-07 18:54:23 ffff880827263098 ffff8808329f9fd8 000000000000fb88 ffff880827263098 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:23 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:23 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:23 [] ? wq_barrier_func+0x0/0x20 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/10 S 000000000000000a 0 3057 2 0x00000000 2014-01-07 18:54:23 ffff88083215be30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:23 ffff88083215bda0 ffffffff81091472 ffff88083215bdb0 ffffffff810914cf 2014-01-07 18:54:23 ffff880832f6c638 ffff88083215bfd8 000000000000fb88 ffff880832f6c638 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? queue_work_on+0x42/0x60 2014-01-07 18:54:23 [] ? queue_work+0x1f/0x30 2014-01-07 18:54:23 [] ? process_queued_ios+0x0/0x290 [dm_multipath] 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/11 S 000000000000000b 0 3058 2 0x00000000 2014-01-07 18:54:23 ffff880832fede30 0000000000000046 ffff880832feddf8 ffff880832feddf4 2014-01-07 18:54:23 ffff880832fedda0 ffff88083fe82600 ffff880044636740 00000000000005d6 2014-01-07 18:54:23 ffff880828d925f8 ffff880832fedfd8 000000000000fb88 ffff880828d925f8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? process_queued_ios+0x0/0x290 [dm_multipath] 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/12 S 000000000000000c 0 3059 2 0x00000000 2014-01-07 18:54:23 ffff880832ea7e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff880828e4e5f8 ffff880832ea7fd8 000000000000fb88 ffff880828e4e5f8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/13 S 000000000000000d 0 3060 2 0x00000000 2014-01-07 18:54:23 ffff880832095e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff880832f23ab8 ffff880832095fd8 000000000000fb88 ffff880832f23ab8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/14 S 000000000000000e 0 3061 2 0x00000000 2014-01-07 18:54:23 ffff8808270b9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff88082710e5f8 ffff8808270b9fd8 000000000000fb88 ffff88082710e5f8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpathd/15 S 000000000000000f 0 3062 2 0x00000000 2014-01-07 18:54:23 ffff880832b7be30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff880828319ab8 ffff880832b7bfd8 000000000000fb88 ffff880828319ab8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 kmpath_handle S 0000000000000005 0 3079 2 0x00000000 2014-01-07 18:54:23 ffff8808323cfe30 0000000000000046 ffff8808327a1148 ffff88080e0680b8 2014-01-07 18:54:23 ffff8808323cfdb0 ffffffff812797b7 ffff8808327a1000 ffff8808327a1000 2014-01-07 18:54:23 ffff8808329d9ab8 ffff8808323cffd8 000000000000fb88 ffff8808329d9ab8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? kobject_put+0x27/0x60 2014-01-07 18:54:23 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:23 [] ? activate_path+0x0/0x30 [dm_multipath] 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/0 S 0000000000000000 0 4429 2 0x00000000 2014-01-07 18:54:23 ffff880832595e30 0000000000000046 ffff880832595da0 ffffffffa02ec2fa 2014-01-07 18:54:23 ffff880832595fd8 ffff880831f78e00 ffff880832595e30 ffffffffa03184ce 2014-01-07 18:54:23 ffff880827311058 ffff880832595fd8 000000000000fb88 ffff880827311058 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? ib_create_ah+0x1a/0x40 [ib_core] 2014-01-07 18:54:23 [] ? update_sm_ah+0x10e/0x1b0 [ib_sa] 2014-01-07 18:54:23 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:23 [] ? update_sm_ah+0x0/0x1b0 [ib_sa] 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/1 S 0000000000000001 0 4430 2 0x00000000 2014-01-07 18:54:23 ffff8808342f7e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff880831071af8 ffff8808342f7fd8 000000000000fb88 ffff880831071af8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/2 S 0000000000000002 0 4431 2 0x00000000 2014-01-07 18:54:23 ffff8808310d3e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff880833fc5ab8 ffff8808310d3fd8 000000000000fb88 ffff880833fc5ab8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/3 S 0000000000000003 0 4432 2 0x00000000 2014-01-07 18:54:23 ffff880832735e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:23 ffff88082710fab8 ffff880832735fd8 000000000000fb88 ffff88082710fab8 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/4 S 0000000000000004 0 4433 2 0x00000000 2014-01-07 18:54:23 ffff880832733e30 0000000000000046 ffff880832733db0 ffff8808150b4c00 2014-01-07 18:54:23 000000000000007f ffff8808342d8000 ffff880832733e10 0000000000000286 2014-01-07 18:54:23 ffff880834341098 ffff880832733fd8 000000000000fb88 ffff880834341098 2014-01-07 18:54:23 Call Trace: 2014-01-07 18:54:23 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:23 [] ? ib_cache_task+0x0/0x30 [ib_core] 2014-01-07 18:54:23 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:23 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:23 [] kthread+0x96/0xa0 2014-01-07 18:54:23 [] child_rip+0xa/0x20 2014-01-07 18:54:23 [] ? kthread+0x0/0xa0 2014-01-07 18:54:23 [] ? child_rip+0x0/0x20 2014-01-07 18:54:23 infiniband/5 S 0000000000000005 0 4434 2 0x00000000 2014-01-07 18:54:23 ffff880832fd7e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:23 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff8808335e7058 ffff880832fd7fd8 000000000000fb88 ffff8808335e7058 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/6 S 0000000000000006 0 4435 2 0x00000000 2014-01-07 18:54:24 ffff88082723de30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff8808343efab8 ffff88082723dfd8 000000000000fb88 ffff8808343efab8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/7 S 0000000000000007 0 4436 2 0x00000000 2014-01-07 18:54:24 ffff8808270c1e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880833fc5058 ffff8808270c1fd8 000000000000fb88 ffff880833fc5058 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/8 S 0000000000000008 0 4437 2 0x00000000 2014-01-07 18:54:24 ffff880834331e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880831d1daf8 ffff880834331fd8 000000000000fb88 ffff880831d1daf8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/9 S 0000000000000009 0 4438 2 0x00000000 2014-01-07 18:54:24 ffff8808343e1e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff8808343865f8 ffff8808343e1fd8 000000000000fb88 ffff8808343865f8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/10 S 000000000000000a 0 4439 2 0x00000000 2014-01-07 18:54:24 ffff8808342dde30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff8808343a3ab8 ffff8808342ddfd8 000000000000fb88 ffff8808343a3ab8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/11 S 000000000000000b 0 4440 2 0x00000000 2014-01-07 18:54:24 ffff880831215e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880834395098 ffff880831215fd8 000000000000fb88 ffff880834395098 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/12 S 000000000000000c 0 4441 2 0x00000000 2014-01-07 18:54:24 ffff880832709e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff88082836a5f8 ffff880832709fd8 000000000000fb88 ffff88082836a5f8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/13 S 000000000000000d 0 4442 2 0x00000000 2014-01-07 18:54:24 ffff880832705e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880832810638 ffff880832705fd8 000000000000fb88 ffff880832810638 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/14 S 000000000000000e 0 4443 2 0x00000000 2014-01-07 18:54:24 ffff88082722be30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880834387ab8 ffff88082722bfd8 000000000000fb88 ffff880834387ab8 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 infiniband/15 S 000000000000000f 0 4444 2 0x00000000 2014-01-07 18:54:24 ffff880831bf7e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880834387058 ffff880831bf7fd8 000000000000fb88 ffff880834387058 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 ib_mcast S 0000000000000000 0 4445 2 0x00000000 2014-01-07 18:54:24 ffff880827233e30 0000000000000046 ffff880827233e30 ffffffffa031a73d 2014-01-07 18:54:24 ffff880800000bb8 ffffffff000000d0 ffffffffa031ae80 ffff8808342e96c0 2014-01-07 18:54:24 ffff88083435a638 ffff880827233fd8 000000000000fb88 ffff88083435a638 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? mcast_work_handler+0xed/0x830 [ib_sa] 2014-01-07 18:54:24 [] ? join_handler+0x0/0x200 [ib_sa] 2014-01-07 18:54:24 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:24 [] ? mcast_work_handler+0x0/0x830 [ib_sa] 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 mlx4_ib S 0000000000000009 0 4446 2 0x00000000 2014-01-07 18:54:24 ffff88082722fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff880831d7d058 ffff88082722ffd8 000000000000fb88 ffff880831d7d058 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 mlx4_ib_mcg S 0000000000000009 0 4447 2 0x00000000 2014-01-07 18:54:24 ffff88083200fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:24 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:24 ffff88082830f098 ffff88083200ffd8 000000000000fb88 ffff88082830f098 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:24 [] ? child_rip+0x0/0x20 2014-01-07 18:54:24 ib_mad1 S 0000000000000007 0 4448 2 0x00000000 2014-01-07 18:54:24 ffff880831bade30 0000000000000046 0000000000000000 ffff8808327d7740 2014-01-07 18:54:24 ffff8808327d7558 ffff880000000001 ffff88007f2e9900 0000000000000000 2014-01-07 18:54:24 ffff880831a4f098 ffff880831badfd8 000000000000fb88 ffff880831a4f098 2014-01-07 18:54:24 Call Trace: 2014-01-07 18:54:24 [] ? ib_mad_completion_handler+0x0/0x8c0 [ib_mad] 2014-01-07 18:54:24 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:24 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:24 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:24 [] kthread+0x96/0xa0 2014-01-07 18:54:24 [] child_rip+0xa/0x20 2014-01-07 18:54:24 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_mad1 S 0000000000000000 0 4449 2 0x00000000 2014-01-07 18:54:25 ffff880832b61e30 0000000000000046 0000000000000000 ffff8800b89fe100 2014-01-07 18:54:25 0000000000000286 ffff8808315829d8 ffff8808315829c0 0000008000000000 2014-01-07 18:54:25 ffff8808319bd098 ffff880832b61fd8 000000000000fb88 ffff8808319bd098 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:25 [] ? ib_mad_completion_handler+0x0/0x8c0 [ib_mad] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_mad2 S 0000000000000004 0 4450 2 0x00000000 2014-01-07 18:54:25 ffff880831903e30 0000000000000046 0000000000000000 ffff8800ba9aad00 2014-01-07 18:54:25 0000000000000286 ffff880833cdca58 ffff880833cdca40 0000008000000000 2014-01-07 18:54:25 ffff880828e4daf8 ffff880831903fd8 000000000000fb88 ffff880828e4daf8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? ib_mad_completion_handler+0x0/0x8c0 [ib_mad] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_addr S 0000000000000009 0 4460 2 0x00000000 2014-01-07 18:54:25 ffff8808323b5e30 0000000000000046 0000000000000000 ffffffff81091b89 2014-01-07 18:54:25 0000000000000000 0000000000000286 ffffffffa003ed80 0000000000005a38 2014-01-07 18:54:25 ffff880833e97ab8 ffff8808323b5fd8 000000000000fb88 ffff880833e97ab8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? queue_delayed_work_on+0xb9/0x120 2014-01-07 18:54:25 [] ? process_req+0x0/0x1a0 [ib_addr] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 iw_cm_wq S 0000000000000009 0 4464 2 0x00000000 2014-01-07 18:54:25 ffff880833c41e30 0000000000000046 ffff880833c41df8 ffff880833c41df4 2014-01-07 18:54:25 0000000000000000 ffff88083fe82200 ffff880044636740 00000000000006de 2014-01-07 18:54:25 ffff8808319b3058 ffff880833c41fd8 000000000000fb88 ffff8808319b3058 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/0 S 0000000000000000 0 4468 2 0x00000000 2014-01-07 18:54:25 ffff880832247e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:25 ffff8807581fc670 ffffe8f7e780b348 ffff880832247de0 ffffffffa0110ce5 2014-01-07 18:54:25 ffff8808319b25f8 ffff880832247fd8 000000000000fb88 ffff8808319b25f8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_rep_handler+0x345/0x600 [ib_cm] 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/1 S 0000000000000001 0 4469 2 0x00000000 2014-01-07 18:54:25 ffff880832391e30 0000000000000046 0000000000000000 ffff880832391df4 2014-01-07 18:54:25 ffff880700000000 ffff88083fe81200 ffff880044616740 00000000000074ed 2014-01-07 18:54:25 ffff880833fcf098 ffff880832391fd8 000000000000fb88 ffff880833fcf098 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/2 S 0000000000000002 0 4470 2 0x00000000 2014-01-07 18:54:25 ffff8808323a1e30 0000000000000046 0000000000000000 ffff88078538fe80 2014-01-07 18:54:25 ffff8800bd5a64e8 ffff8800bd5a6538 ffff880044636740 ffff880700000000 2014-01-07 18:54:25 ffff880833e965f8 ffff8808323a1fd8 000000000000fb88 ffff880833e965f8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/3 S 0000000000000003 0 4471 2 0x00000000 2014-01-07 18:54:25 ffff8808320f5e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:25 ffff88007fbe04e8 ffff88007fbe0538 0000000000000000 0000000000000000 2014-01-07 18:54:25 ffff880831a45058 ffff8808320f5fd8 000000000000fb88 ffff880831a45058 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/4 S 0000000000000004 0 4472 2 0x00000000 2014-01-07 18:54:25 ffff880831e0de30 0000000000000046 0000000000000000 ffff880833762600 2014-01-07 18:54:25 0000000000000000 ffff880833762600 0000000000000000 ffffffffa0110fa0 2014-01-07 18:54:25 ffff880832589058 ffff880831e0dfd8 000000000000fb88 ffff880832589058 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/5 S 0000000000000005 0 4473 2 0x00000000 2014-01-07 18:54:25 ffff8808324e7e30 0000000000000046 0000000000000000 ffff88007f92b800 2014-01-07 18:54:25 ffff880831267a70 ffff88007f92b800 0000000000000000 ffffffffa0110fa0 2014-01-07 18:54:25 ffff8808343ef058 ffff8808324e7fd8 000000000000fb88 ffff8808343ef058 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/6 S 0000000000000006 0 4474 2 0x00000000 2014-01-07 18:54:25 ffff8808271b5e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:25 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:25 ffff880832f6daf8 ffff8808271b5fd8 000000000000fb88 ffff880832f6daf8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/7 S 0000000000000007 0 4475 2 0x00000000 2014-01-07 18:54:25 ffff88082718de30 0000000000000046 ffff88082718ddf8 ffff88082718ddf4 2014-01-07 18:54:25 0000000000000000 ffff88083fe81e00 ffff8800446b6740 0000000000000400 2014-01-07 18:54:25 ffff8808325d3098 ffff88082718dfd8 000000000000fb88 ffff8808325d3098 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/8 S 0000000000000008 0 4476 2 0x00000000 2014-01-07 18:54:25 ffff88082713de30 0000000000000046 0000000000000000 ffff8800b8a98800 2014-01-07 18:54:25 0000000000000000 ffff8800b8a98800 0000000000000000 ffffffffa0110fa0 2014-01-07 18:54:25 ffff880827262638 ffff88082713dfd8 000000000000fb88 ffff880827262638 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] ? cm_work_handler+0x0/0x1206 [ib_cm] 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:25 [] ? child_rip+0x0/0x20 2014-01-07 18:54:25 ib_cm/9 S 0000000000000009 0 4477 2 0x00000000 2014-01-07 18:54:25 ffff880827115e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:25 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:25 ffff880831abe5f8 ffff880827115fd8 000000000000fb88 ffff880831abe5f8 2014-01-07 18:54:25 Call Trace: 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:25 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:25 [] kthread+0x96/0xa0 2014-01-07 18:54:25 [] child_rip+0xa/0x20 2014-01-07 18:54:25 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/10 S 000000000000000a 0 4478 2 0x00000000 2014-01-07 18:54:26 ffff8808327ede30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff880833585098 ffff8808327edfd8 000000000000fb88 ffff880833585098 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/11 S 000000000000000b 0 4479 2 0x00000000 2014-01-07 18:54:26 ffff8808327c5e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff8808335fdab8 ffff8808327c5fd8 000000000000fb88 ffff8808335fdab8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/12 S 000000000000000c 0 4480 2 0x00000000 2014-01-07 18:54:26 ffff88083279be30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff88083191d058 ffff88083279bfd8 000000000000fb88 ffff88083191d058 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/13 S 000000000000000d 0 4481 2 0x00000000 2014-01-07 18:54:26 ffff880832f99e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff8808319d3af8 ffff880832f99fd8 000000000000fb88 ffff8808319d3af8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/14 S 000000000000000e 0 4482 2 0x00000000 2014-01-07 18:54:26 ffff880832749e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff880831a65af8 ffff880832749fd8 000000000000fb88 ffff880831a65af8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ib_cm/15 S 000000000000000f 0 4483 2 0x00000000 2014-01-07 18:54:26 ffff880832727e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:26 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:26 ffff88082885baf8 ffff880832727fd8 000000000000fb88 ffff88082885baf8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 rdma_cm S 0000000000000007 0 4487 2 0x00000000 2014-01-07 18:54:26 ffff880831d6be30 0000000000000046 0000000000000000 0000060501000038 2014-01-07 18:54:26 ffff880831d6be00 0000000000000286 ffff8808343fb578 ffff8807c9e05c70 2014-01-07 18:54:26 ffff8808343fbaf8 ffff880831d6bfd8 000000000000fb88 ffff8808343fbaf8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? cma_work_handler+0x0/0xb0 [rdma_cm] 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 ipoib S 0000000000000005 0 4503 2 0x00000000 2014-01-07 18:54:26 ffff880831c6be30 0000000000000046 0000000000000000 ffff880831c6bdf4 2014-01-07 18:54:26 0000000000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:54:26 ffff880833c03ab8 ffff880831c6bfd8 000000000000fb88 ffff880833c03ab8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? ipoib_reap_ah+0x0/0x50 [ib_ipoib] 2014-01-07 18:54:26 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:26 [] kthread+0x96/0xa0 2014-01-07 18:54:26 [] child_rip+0xa/0x20 2014-01-07 18:54:26 [] ? kthread+0x0/0xa0 2014-01-07 18:54:26 [] ? child_rip+0x0/0x20 2014-01-07 18:54:26 multipathd S 000000000000000a 0 4537 1 0x00000000 2014-01-07 18:54:26 ffff8810320b7bf8 0000000000000086 0000000000000246 0000000000000004 2014-01-07 18:54:26 ffffffffffffffff 0000000000000002 ffff8810320b7b88 0000000030cc4ec0 2014-01-07 18:54:26 ffff88103185d098 ffff8810320b7fd8 000000000000fb88 ffff88103185d098 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:26 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:26 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:26 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:26 [] ? cpumask_next_and+0x29/0x50 2014-01-07 18:54:26 [] do_futex+0x121/0xb60 2014-01-07 18:54:26 [] ? check_preempt_curr_rt+0x7d/0xe0 2014-01-07 18:54:26 [] ? check_preempt_curr+0x7c/0x90 2014-01-07 18:54:26 [] ? wake_up_new_task+0xd9/0x130 2014-01-07 18:54:26 [] ? do_fork+0x133/0x460 2014-01-07 18:54:26 [] sys_futex+0x7b/0x170 2014-01-07 18:54:26 [] ? sys_clone+0x28/0x30 2014-01-07 18:54:26 [] ? stub_clone+0x13/0x20 2014-01-07 18:54:26 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:26 multipathd S 000000000000000b 0 4540 1 0x00000000 2014-01-07 18:54:26 ffff881030adfbf8 0000000000000086 0000000000000000 ffff881030adfbbc 2014-01-07 18:54:26 0000000000000000 ffff88083fe82600 ffff88085c436740 0000000000000319 2014-01-07 18:54:26 ffff881031b8f098 ffff881030adffd8 000000000000fb88 ffff881031b8f098 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:26 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:26 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:26 [] ? rwsem_down_read_failed+0x26/0x30 2014-01-07 18:54:26 [] do_futex+0x121/0xb60 2014-01-07 18:54:26 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:26 [] ? sys_sendto+0x185/0x190 2014-01-07 18:54:26 [] sys_futex+0x7b/0x170 2014-01-07 18:54:26 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:26 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:26 multipathd S 000000000000000a 0 4542 1 0x00000000 2014-01-07 18:54:26 ffff8810318a7a58 0000000000000086 ffff8810318a7a20 ffff8810318a7a1c 2014-01-07 18:54:26 ffff88000000000a ffff88083fe82400 ffff880044676740 0000000000000607 2014-01-07 18:54:26 ffff881030357ab8 ffff8810318a7fd8 000000000000fb88 ffff881030357ab8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:26 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:26 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:26 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:26 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:26 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:26 [] ? pollwake+0x0/0x60 2014-01-07 18:54:26 [] ? do_wp_page+0x493/0x920 2014-01-07 18:54:26 [] ? put_rpccred+0x50/0x150 [sunrpc] 2014-01-07 18:54:26 [] ? handle_pte_fault+0x2cd/0xb50 2014-01-07 18:54:26 [] ? inotify_d_instantiate+0x2a/0x60 2014-01-07 18:54:26 [] ? __d_instantiate+0xbb/0x110 2014-01-07 18:54:26 [] ? fsnotify_create+0x60/0x80 2014-01-07 18:54:26 [] ? handle_mm_fault+0x23a/0x310 2014-01-07 18:54:26 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:26 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:26 [] ? poll_select_set_timeout+0x8d/0xa0 2014-01-07 18:54:26 [] sys_poll+0x71/0x100 2014-01-07 18:54:26 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:26 multipathd S 0000000000000008 0 4544 1 0x00000000 2014-01-07 18:54:26 ffff881031c3fe68 0000000000000086 ffff881031c3fe58 ffffffff8109bc05 2014-01-07 18:54:26 00007fff00000008 0000000100002285 000000000000001d ffff880800000000 2014-01-07 18:54:26 ffff881037d47af8 ffff881031c3ffd8 000000000000fb88 ffff881037d47af8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:26 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:26 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:26 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:26 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:26 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:26 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:26 multipathd S 000000000000000a 0 4545 1 0x00000000 2014-01-07 18:54:26 ffff881031da9998 0000000000000086 0000000000000000 ffff881031da995c 2014-01-07 18:54:26 ffff88103038b540 ffff88083fe82400 ffff880044616740 0000000000000617 2014-01-07 18:54:26 ffff88103038baf8 ffff881031da9fd8 000000000000fb88 ffff88103038baf8 2014-01-07 18:54:26 Call Trace: 2014-01-07 18:54:26 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:26 [] __skb_recv_datagram+0x237/0x290 2014-01-07 18:54:26 [] ? receiver_wake_function+0x0/0x20 2014-01-07 18:54:26 [] ? __kfree_skb+0x47/0xa0 2014-01-07 18:54:26 [] skb_recv_datagram+0x24/0x30 2014-01-07 18:54:26 [] unix_dgram_recvmsg+0x86/0x380 2014-01-07 18:54:26 [] sock_recvmsg+0x133/0x160 2014-01-07 18:54:26 [] ? ptep_set_access_flags+0x6d/0x70 2014-01-07 18:54:26 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:26 [] ? enqueue_pushable_task+0x73/0x90 2014-01-07 18:54:26 [] ? enqueue_task_rt+0x5e/0x70 2014-01-07 18:54:27 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:27 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:27 [] __sys_recvmsg+0x13c/0x2f0 2014-01-07 18:54:27 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:27 [] ? release_sock+0xce/0xe0 2014-01-07 18:54:27 [] sys_recvmsg+0x49/0x90 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 0000000000000009 0 4546 1 0x00000000 2014-01-07 18:54:27 ffff8810300b1bf8 0000000000000086 0000000000000000 00000000f41ce31c 2014-01-07 18:54:27 ffff8810300b1b98 ffffc900209e11c3 ffffffffa02d24cc ffffffffa02d24ca 2014-01-07 18:54:27 ffff881030fb1ab8 ffff8810300b1fd8 000000000000fb88 ffff881030fb1ab8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:27 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:27 [] ? scnprintf+0x34/0x40 2014-01-07 18:54:27 [] ? free_hot_page+0x2f/0x60 2014-01-07 18:54:27 [] do_futex+0x121/0xb60 2014-01-07 18:54:27 [] ? ctl_ioctl+0x1ca/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? table_status+0x0/0x90 [dm_mod] 2014-01-07 18:54:27 [] ? dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] ? vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] ? _atomic_dec_and_lock+0x55/0x80 2014-01-07 18:54:27 [] ? do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_futex+0x7b/0x170 2014-01-07 18:54:27 [] ? sys_ioctl+0x8e/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 0000000000000009 0 5444 1 0x00000000 2014-01-07 18:54:27 ffff881031b8dd08 0000000000000086 0000000000000000 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff8810302dfaf8 ffff881031b8dfd8 000000000000fb88 ffff8810302dfaf8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 0000000000000008 0 5454 1 0x00000000 2014-01-07 18:54:27 ffff8810327d3d08 0000000000000086 ffff8810327d3dc8 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff881030cb7af8 ffff8810327d3fd8 000000000000fb88 ffff881030cb7af8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 0000000000000009 0 5460 1 0x00000000 2014-01-07 18:54:27 ffff88103324bd08 0000000000000086 ffff88103324bdc8 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff881031c15098 ffff88103324bfd8 000000000000fb88 ffff881031c15098 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 000000000000000a 0 5473 1 0x00000000 2014-01-07 18:54:27 ffff881031a63d08 0000000000000086 ffff881031a63dc8 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff88102f03a5f8 ffff881031a63fd8 000000000000fb88 ffff88102f03a5f8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 000000000000000a 0 5484 1 0x00000000 2014-01-07 18:54:27 ffff881031b8bd08 0000000000000086 0000000000000018 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff881031b89af8 ffff881031b8bfd8 000000000000fb88 ffff881031b89af8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] ? rwsem_wake+0x75/0x170 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 multipathd S 0000000000000008 0 5489 1 0x00000000 2014-01-07 18:54:27 ffff881030255d08 0000000000000086 ffff881030255dc8 ffffffffa02b67b1 2014-01-07 18:54:27 0000001700000004 0000400000000006 0000000000000138 0000020400000000 2014-01-07 18:54:27 ffff881030a83ab8 ffff881030255fd8 000000000000fb88 ffff881030a83ab8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? copy_params+0xc1/0x140 [dm_mod] 2014-01-07 18:54:27 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:27 [] dm_wait_event+0x91/0xe0 [dm_mod] 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] dev_wait+0x4c/0xc0 [dm_mod] 2014-01-07 18:54:27 [] ctl_ioctl+0x1b4/0x270 [dm_mod] 2014-01-07 18:54:27 [] ? dev_wait+0x0/0xc0 [dm_mod] 2014-01-07 18:54:27 [] dm_ctl_ioctl+0x13/0x20 [dm_mod] 2014-01-07 18:54:27 [] vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] sys_ioctl+0x81/0xa0 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 syslog-ng S 0000000000000000 0 4738 1 0x00000000 2014-01-07 18:54:27 ffff88083350fe78 0000000000000086 0000000000000000 0000000000000028 2014-01-07 18:54:27 ffff8808327762c0 00007ffff6823040 ffff8808320d7210 0000000000000014 2014-01-07 18:54:27 ffff8808272865f8 ffff88083350ffd8 000000000000fb88 ffff8808272865f8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] do_wait+0x1e4/0x240 2014-01-07 18:54:27 [] sys_wait4+0xa3/0x100 2014-01-07 18:54:27 [] ? child_wait_callback+0x0/0x70 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 syslog-ng S 0000000000000000 0 4739 4738 0x00000000 2014-01-07 18:54:27 ffff881030d87a58 0000000000000086 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:27 ffff881000000000 0000000181485c38 ffff88083feba9f8 ffff880831212f10 2014-01-07 18:54:27 ffff8810341d5098 ffff881030d87fd8 000000000000fb88 ffff8810341d5098 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:27 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:27 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:27 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:27 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:27 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:27 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? pollwake+0x0/0x60 2014-01-07 18:54:27 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:27 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:27 [] ? poll_select_set_timeout+0x8d/0xa0 2014-01-07 18:54:27 [] sys_poll+0x71/0x100 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 shutdown-hot S 0000000000000002 0 4740 4739 0x00000000 2014-01-07 18:54:27 ffff880833595ca8 0000000000000086 0000000000000000 ffffffff81119f27 2014-01-07 18:54:27 0000000000470000 ffff881030a394a8 ffff880833595ca8 ffffffff811436a9 2014-01-07 18:54:27 ffff880833cda5f8 ffff880833595fd8 000000000000fb88 ffff880833cda5f8 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? unlock_page+0x27/0x30 2014-01-07 18:54:27 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:27 [] pipe_wait+0x5b/0x80 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] ? mutex_lock+0x1e/0x50 2014-01-07 18:54:27 [] pipe_read+0x3e6/0x4e0 2014-01-07 18:54:27 [] do_sync_read+0xfa/0x140 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] ? vfs_ioctl+0x22/0xa0 2014-01-07 18:54:27 [] ? do_vfs_ioctl+0x84/0x580 2014-01-07 18:54:27 [] ? security_file_permission+0x16/0x20 2014-01-07 18:54:27 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:27 [] sys_read+0x51/0x90 2014-01-07 18:54:27 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:27 kondemand/0 S 0000000000000000 0 4762 2 0x00000000 2014-01-07 18:54:27 ffff8808150a7e30 0000000000000046 0000000000000000 ffff8808150a7df4 2014-01-07 18:54:27 ffff880800000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:27 ffff880827185058 ffff8808150a7fd8 000000000000fb88 ffff880827185058 2014-01-07 18:54:27 Call Trace: 2014-01-07 18:54:27 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:27 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:27 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:27 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:27 [] kthread+0x96/0xa0 2014-01-07 18:54:27 [] child_rip+0xa/0x20 2014-01-07 18:54:27 [] ? kthread+0x0/0xa0 2014-01-07 18:54:27 [] ? child_rip+0x0/0x20 2014-01-07 18:54:27 kondemand/1 S 0000000000000001 0 4763 2 0x00000000 2014-01-07 18:54:27 ffff880815067e30 0000000000000046 0000000000000000 ffff880833e3f3c0 2014-01-07 18:54:27 ffff880815067e30 ffffffffa03061fc ffff880000002326 00000001329949b5 2014-01-07 18:54:28 ffff880832561af8 ffff880815067fd8 000000000000fb88 ffff880832561af8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/2 S 0000000000000002 0 4764 2 0x00000000 2014-01-07 18:54:28 ffff880832a1fe30 0000000000000046 0000000000000000 ffff880832a1fdf4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:28 ffff8808343fb098 ffff880832a1ffd8 000000000000fb88 ffff8808343fb098 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/3 S 0000000000000003 0 4765 2 0x00000000 2014-01-07 18:54:28 ffff880827289e30 0000000000000046 0000000000000000 ffff880827289df4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:28 ffff8808328c65f8 ffff880827289fd8 000000000000fb88 ffff8808328c65f8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/4 S 0000000000000004 0 4766 2 0x00000000 2014-01-07 18:54:28 ffff8808329f5e30 0000000000000046 0000000000000000 ffff8808329f5df4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:28 ffff880833503098 ffff8808329f5fd8 000000000000fb88 ffff880833503098 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/5 S 0000000000000005 0 4767 2 0x00000000 2014-01-07 18:54:28 ffff8808342b3e30 0000000000000046 0000000000011400 ffff8808270bf580 2014-01-07 18:54:28 ffff8808342b3e30 ffffffffa03061fc ffff880000003ea2 00000001339837c2 2014-01-07 18:54:28 ffff8808335f2638 ffff8808342b3fd8 000000000000fb88 ffff8808335f2638 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/6 S 0000000000000006 0 4768 2 0x00000000 2014-01-07 18:54:28 ffff8808283b9e30 0000000000000046 0000000000000000 ffff8808283b9df4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:28 ffff8808334ec638 ffff8808283b9fd8 000000000000fb88 ffff8808334ec638 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/7 S 0000000000000007 0 4769 2 0x00000000 2014-01-07 18:54:28 ffff880832a8de30 0000000000000046 0000000000000000 ffff880832a8ddf4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:54:28 ffff880833585af8 ffff880832a8dfd8 000000000000fb88 ffff880833585af8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/8 S 0000000000000008 0 4770 2 0x00000000 2014-01-07 18:54:28 ffff880833fc7e30 0000000000000046 0000000000000000 ffff880833fc7df4 2014-01-07 18:54:28 ffff880833fc7e30 ffff88083fe82000 ffff880044636740 0000000000000406 2014-01-07 18:54:28 ffff8808319bc638 ffff880833fc7fd8 000000000000fb88 ffff8808319bc638 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/9 S 0000000000000009 0 4771 2 0x00000000 2014-01-07 18:54:28 ffff880832a2fe30 0000000000000046 ffff880832a2fdf8 ffff880832a2fdf4 2014-01-07 18:54:28 ffff880832a2fe30 ffff88083fe82200 ffff880044636740 00000001375076ec 2014-01-07 18:54:28 ffff8808319065f8 ffff880832a2ffd8 000000000000fb88 ffff8808319065f8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/10 S 000000000000000a 0 4772 2 0x00000000 2014-01-07 18:54:28 ffff88081518fe30 0000000000000046 0000000000000000 ffff88081518fdf4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:54:28 ffff88083274b098 ffff88081518ffd8 000000000000fb88 ffff88083274b098 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/11 S 000000000000000b 0 4773 2 0x00000000 2014-01-07 18:54:28 ffff8808334e5e30 0000000000000046 0000000000000000 ffff8808334e5df4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe82600 ffff880044636740 00000000000001ff 2014-01-07 18:54:28 ffff8808318f25f8 ffff8808334e5fd8 000000000000fb88 ffff8808318f25f8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/12 S 000000000000000c 0 4774 2 0x00000000 2014-01-07 18:54:28 ffff880834fe5e30 0000000000000046 ffff880834fe5df8 ffff880834fe5df4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:54:28 ffff8808319a7098 ffff880834fe5fd8 000000000000fb88 ffff8808319a7098 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/13 S 000000000000000d 0 4775 2 0x00000000 2014-01-07 18:54:28 ffff88083293de30 0000000000000046 0000000000000000 ffff88083293ddf4 2014-01-07 18:54:28 ffff880800000000 ffff88083fe82a00 ffff880044636740 00000000000003ff 2014-01-07 18:54:28 ffff880831a39098 ffff88083293dfd8 000000000000fb88 ffff880831a39098 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/14 S 000000000000000e 0 4776 2 0x00000000 2014-01-07 18:54:28 ffff880831a5be30 0000000000000046 0000000000011400 ffff880833e3f4c0 2014-01-07 18:54:28 ffff880831a5be30 ffffffffa03061fc ffff880044636740 00000000000001ff 2014-01-07 18:54:28 ffff880828e685f8 ffff880831a5bfd8 000000000000fb88 ffff880828e685f8 2014-01-07 18:54:28 Call Trace: 2014-01-07 18:54:28 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:28 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:28 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:28 [] kthread+0x96/0xa0 2014-01-07 18:54:28 [] child_rip+0xa/0x20 2014-01-07 18:54:28 [] ? kthread+0x0/0xa0 2014-01-07 18:54:28 [] ? child_rip+0x0/0x20 2014-01-07 18:54:28 kondemand/15 S 000000000000000f 0 4777 2 0x00000000 2014-01-07 18:54:28 ffff880827111e30 0000000000000046 0000000000011400 ffff8808342e90c0 2014-01-07 18:54:28 ffff880827111e30 ffffffffa03061fc ffff880000002afc 0000000000000406 2014-01-07 18:54:29 ffff8808327c3098 ffff880827111fd8 000000000000fb88 ffff8808327c3098 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? do_dbs_timer+0x27c/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:29 [] ? do_dbs_timer+0x0/0x3bc [cpufreq_ondemand] 2014-01-07 18:54:29 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:29 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:29 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:29 [] kthread+0x96/0xa0 2014-01-07 18:54:29 [] child_rip+0xa/0x20 2014-01-07 18:54:29 [] ? kthread+0x0/0xa0 2014-01-07 18:54:29 [] ? child_rip+0x0/0x20 2014-01-07 18:54:29 irqbalance S 0000000000000003 0 4818 1 0x00000000 2014-01-07 18:54:29 ffff8808150afe68 0000000000000086 ffff8808150afe58 ffffffff8109bc05 2014-01-07 18:54:29 ffff880800000003 00000001f8000000 ffff8808150afde8 ffffffff8114d1e8 2014-01-07 18:54:29 ffff880831a4e638 ffff8808150affd8 000000000000fb88 ffff880831a4e638 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:29 [] ? anon_vma_chain_free+0x18/0x20 2014-01-07 18:54:29 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:29 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:29 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:29 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:29 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 rpcbind S 0000000000000008 0 4832 1 0x00000000 2014-01-07 18:54:29 ffff8808151a9a58 0000000000000082 0000000000000000 ffff8808151a9a1c 2014-01-07 18:54:29 0000000000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:29 ffff880831ce6638 ffff8808151a9fd8 000000000000fb88 ffff880831ce6638 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:29 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:29 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? fsnotify_clear_marks_by_inode+0x32/0xf0 2014-01-07 18:54:29 [] ? sock_destroy_inode+0x19/0x20 2014-01-07 18:54:29 [] ? __d_free+0x3f/0x60 2014-01-07 18:54:29 [] ? d_free+0x58/0x60 2014-01-07 18:54:29 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:29 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:29 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:29 [] ? poll_select_set_timeout+0x8d/0xa0 2014-01-07 18:54:29 [] sys_poll+0x71/0x100 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 rpc.statd S 0000000000000000 0 4850 1 0x00000000 2014-01-07 18:54:29 ffff881031be7908 0000000000000082 0000000000000000 ffffffff813a1a35 2014-01-07 18:54:29 0000000000000034 0000000000000042 0000000000000000 ffff880828b5f6c0 2014-01-07 18:54:29 ffff88103236e5f8 ffff881031be7fd8 000000000000fb88 ffff88103236e5f8 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? loopback_xmit+0x75/0xa0 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:29 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:29 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_select+0x57c/0x6c0 2014-01-07 18:54:29 [] ? udp_send_skb+0x119/0x3d0 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? dput+0x9a/0x150 2014-01-07 18:54:29 [] ? path_to_nameidata+0x25/0x60 2014-01-07 18:54:29 [] ? __link_path_walk+0x768/0x1030 2014-01-07 18:54:29 [] ? handle_pte_fault+0x2cd/0xb50 2014-01-07 18:54:29 [] ? move_addr_to_kernel+0x64/0x70 2014-01-07 18:54:29 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:54:29 [] ? cp_new_stat+0xe4/0x100 2014-01-07 18:54:29 [] sys_select+0x47/0x110 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 rpc.idmapd S 0000000000000009 0 4871 1 0x00000000 2014-01-07 18:54:29 ffff8808150dfda8 0000000000000082 ffff880828318aa0 ffffffff81096da0 2014-01-07 18:54:29 ffff8808150dfd08 ffff8808150dfd08 ffff8808150dfd68 ffff8808150dfd88 2014-01-07 18:54:29 ffff880828319058 ffff8808150dffd8 000000000000fb88 ffff880828319058 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:29 [] ? ep_scan_ready_list+0x194/0x1a0 2014-01-07 18:54:29 [] ep_poll+0x2ad/0x330 2014-01-07 18:54:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:29 [] sys_epoll_wait+0xc5/0xe0 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 dbus-daemon S 0000000000000001 0 4899 1 0x00000000 2014-01-07 18:54:29 ffff880833cf5a58 0000000000000082 ffff880044656740 0000000000000003 2014-01-07 18:54:29 ffff880833cf59e8 ffffffff81055f96 ffff880833cf59f8 ffff880832799500 2014-01-07 18:54:29 ffff8808327c2638 ffff880833cf5fd8 000000000000fb88 ffff8808327c2638 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:29 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:29 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:29 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? do_sync_readv_writev+0xfb/0x140 2014-01-07 18:54:29 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:29 [] ? do_readv_writev+0x169/0x1f0 2014-01-07 18:54:29 [] ? __d_free+0x3f/0x60 2014-01-07 18:54:29 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:29 [] ? __fput+0x1a1/0x210 2014-01-07 18:54:29 [] sys_poll+0x71/0x100 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 hald S 0000000000000001 0 4923 1 0x00000000 2014-01-07 18:54:29 ffff8808343e3a58 0000000000000086 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:29 ffff880800000001 0000000100000286 ffff8808343e39f8 ffffffff8109b2c1 2014-01-07 18:54:29 ffff880832799ab8 ffff8808343e3fd8 000000000000fb88 ffff880832799ab8 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:29 [] ? lock_hrtimer_base+0x31/0x60 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:29 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:29 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? unmap_region+0x110/0x130 2014-01-07 18:54:29 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:29 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:29 [] ? poll_select_set_timeout+0x8d/0xa0 2014-01-07 18:54:29 [] sys_poll+0x71/0x100 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 hald-runner S 000000000000000a 0 4924 4923 0x00000000 2014-01-07 18:54:29 ffff881030841a58 0000000000000082 0000000000000000 ffff88085c4515e8 2014-01-07 18:54:29 ffff8810308419f8 ffffffff8109ad0b ffff8810308419f8 ffffffff8109b2c1 2014-01-07 18:54:29 ffff881030a7c638 ffff881030841fd8 000000000000fb88 ffff881030a7c638 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? __remove_hrtimer+0x3b/0xb0 2014-01-07 18:54:29 [] ? lock_hrtimer_base+0x31/0x60 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:29 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:29 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:29 [] ? proc_delete_inode+0x0/0x60 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? do_wp_page+0x493/0x920 2014-01-07 18:54:29 [] ? handle_pte_fault+0x2cd/0xb50 2014-01-07 18:54:29 [] ? sock_aio_write+0x0/0x1c0 2014-01-07 18:54:29 [] ? do_sync_readv_writev+0xfb/0x140 2014-01-07 18:54:29 [] ? d_hash_and_lookup+0x83/0xb0 2014-01-07 18:54:29 [] ? handle_mm_fault+0x23a/0x310 2014-01-07 18:54:29 [] ? do_wait+0x17f/0x240 2014-01-07 18:54:29 [] sys_poll+0x71/0x100 2014-01-07 18:54:29 [] ? page_fault+0x25/0x30 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 hald-addon-in S 0000000000000008 0 4953 4924 0x00000000 2014-01-07 18:54:29 ffff880834361a58 0000000000000086 0000000000000000 0000000000000003 2014-01-07 18:54:29 ffff8808343619e8 ffffffff81055f96 ffff8808343619f8 ffff880832799500 2014-01-07 18:54:29 ffff8808327c3af8 ffff880834361fd8 000000000000fb88 ffff8808327c3af8 2014-01-07 18:54:29 Call Trace: 2014-01-07 18:54:29 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:29 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:29 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:29 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:29 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:29 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:29 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? pollwake+0x0/0x60 2014-01-07 18:54:29 [] ? sock_aio_write+0x0/0x1c0 2014-01-07 18:54:29 [] ? do_sync_readv_writev+0xfb/0x140 2014-01-07 18:54:29 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:29 [] ? do_filp_open+0x6ea/0xdc0 2014-01-07 18:54:29 [] ? do_readv_writev+0x169/0x1f0 2014-01-07 18:54:29 [] sys_poll+0x71/0x100 2014-01-07 18:54:29 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:29 hald-addon-ac S 0000000000000006 0 4959 4924 0x00000000 2014-01-07 18:54:29 ffff880828e35e68 0000000000000086 0000000000000000 ffff880828e35e2c 2014-01-07 18:54:30 ffff880800000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:30 ffff880831ce7af8 ffff880828e35fd8 000000000000fb88 ffff880831ce7af8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:30 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:30 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:30 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:30 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:30 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:30 munged S 0000000000000001 0 5012 1 0x00000000 2014-01-07 18:54:30 ffff881033127c98 0000000000000082 0000000000000000 ffffffff810573c5 2014-01-07 18:54:30 0000000000000000 0000000200000002 000000105c416740 00000000000e12d0 2014-01-07 18:54:30 ffff8810303565f8 ffff881033127fd8 000000000000fb88 ffff8810303565f8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? select_idle_sibling+0x95/0x150 2014-01-07 18:54:30 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:30 [] ? cache_grow+0x217/0x320 2014-01-07 18:54:30 [] __skb_recv_datagram+0x237/0x290 2014-01-07 18:54:30 [] ? receiver_wake_function+0x0/0x20 2014-01-07 18:54:30 [] skb_recv_datagram+0x24/0x30 2014-01-07 18:54:30 [] unix_accept+0x74/0x130 2014-01-07 18:54:30 [] sys_accept4+0x155/0x2b0 2014-01-07 18:54:30 [] ? sys_futex+0x7b/0x170 2014-01-07 18:54:30 [] ? security_file_fcntl+0x16/0x20 2014-01-07 18:54:30 [] sys_accept+0x10/0x20 2014-01-07 18:54:30 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:30 munged S 000000000000000f 0 5013 1 0x00000000 2014-01-07 18:54:30 ffff88081526bbf8 0000000000000082 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:30 000000000000000f 000000013fe82e00 ffff88085c4b6740 00000000000002af 2014-01-07 18:54:30 ffff88083113dab8 ffff88081526bfd8 000000000000fb88 ffff88083113dab8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:30 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:30 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:30 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:30 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:30 [] do_futex+0x121/0xb60 2014-01-07 18:54:30 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:30 [] ? rcu_start_gp+0x1be/0x230 2014-01-07 18:54:30 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:30 [] sys_futex+0x7b/0x170 2014-01-07 18:54:30 [] ? __fput+0x1e0/0x210 2014-01-07 18:54:30 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:30 munged S 0000000000000002 0 5014 1 0x00000000 2014-01-07 18:54:30 ffff88081526dbf8 0000000000000082 0000000000000000 ffff88081526dbbc 2014-01-07 18:54:30 ffff880800000000 ffff88083fe81400 ffff8800446f6740 0000000000000207 2014-01-07 18:54:30 ffff880834395af8 ffff88081526dfd8 000000000000fb88 ffff880834395af8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:30 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] do_futex+0x121/0xb60 2014-01-07 18:54:30 [] ? sock_destroy_inode+0x19/0x20 2014-01-07 18:54:30 [] ? __d_free+0x3f/0x60 2014-01-07 18:54:30 [] ? d_free+0x58/0x60 2014-01-07 18:54:30 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:30 [] sys_futex+0x7b/0x170 2014-01-07 18:54:30 [] ? fput+0x25/0x30 2014-01-07 18:54:30 [] ? filp_close+0x5d/0x90 2014-01-07 18:54:30 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:30 munged S 0000000000000002 0 5015 1 0x00000000 2014-01-07 18:54:30 ffff88081526fbf8 0000000000000082 0000000000000000 ffffffff81439c14 2014-01-07 18:54:30 ffff88081526fc48 0000000000000060 ffff88081526fc28 ffffffff814dd379 2014-01-07 18:54:30 ffff880833cd0638 ffff88081526ffd8 000000000000fb88 ffff880833cd0638 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? sock_def_readable+0x44/0x80 2014-01-07 18:54:30 [] ? unix_stream_sendmsg+0x249/0x420 2014-01-07 18:54:30 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:30 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] do_futex+0x121/0xb60 2014-01-07 18:54:30 [] ? sock_destroy_inode+0x19/0x20 2014-01-07 18:54:30 [] ? __d_free+0x3f/0x60 2014-01-07 18:54:30 [] ? d_free+0x58/0x60 2014-01-07 18:54:30 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:30 [] sys_futex+0x7b/0x170 2014-01-07 18:54:30 [] ? fput+0x25/0x30 2014-01-07 18:54:30 [] ? filp_close+0x5d/0x90 2014-01-07 18:54:30 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:30 scsi_tgtd/0 S 0000000000000000 0 5025 2 0x00000000 2014-01-07 18:54:30 ffff8808151d9e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808328d8638 ffff8808151d9fd8 000000000000fb88 ffff8808328d8638 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/1 S 0000000000000001 0 5026 2 0x00000000 2014-01-07 18:54:30 ffff8808151a7e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff880833cc7058 ffff8808151a7fd8 000000000000fb88 ffff880833cc7058 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/2 S 0000000000000002 0 5027 2 0x00000000 2014-01-07 18:54:30 ffff880832211e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808325d2638 ffff880832211fd8 000000000000fb88 ffff8808325d2638 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/3 S 0000000000000003 0 5028 2 0x00000000 2014-01-07 18:54:30 ffff880831d15e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808342825f8 ffff880831d15fd8 000000000000fb88 ffff8808342825f8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/4 S 0000000000000004 0 5029 2 0x00000000 2014-01-07 18:54:30 ffff880815269e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff880834283058 ffff880815269fd8 000000000000fb88 ffff880834283058 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/5 S 0000000000000005 0 5030 2 0x00000000 2014-01-07 18:54:30 ffff880831e71e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808342b1098 ffff880831e71fd8 000000000000fb88 ffff8808342b1098 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/6 S 0000000000000006 0 5031 2 0x00000000 2014-01-07 18:54:30 ffff8808310f1e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808342b1af8 ffff8808310f1fd8 000000000000fb88 ffff8808342b1af8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/7 S 0000000000000007 0 5032 2 0x00000000 2014-01-07 18:54:30 ffff88083266be30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff8808271b2638 ffff88083266bfd8 000000000000fb88 ffff8808271b2638 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] kthread+0x96/0xa0 2014-01-07 18:54:30 [] child_rip+0xa/0x20 2014-01-07 18:54:30 [] ? kthread+0x0/0xa0 2014-01-07 18:54:30 [] ? child_rip+0x0/0x20 2014-01-07 18:54:30 scsi_tgtd/8 S 0000000000000008 0 5033 2 0x00000000 2014-01-07 18:54:30 ffff88081523fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:30 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:30 ffff880831d7dab8 ffff88081523ffd8 000000000000fb88 ffff880831d7dab8 2014-01-07 18:54:30 Call Trace: 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:30 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:30 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/9 S 0000000000000009 0 5034 2 0x00000000 2014-01-07 18:54:31 ffff880831d21e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff8808319e9098 ffff880831d21fd8 000000000000fb88 ffff8808319e9098 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/10 S 000000000000000a 0 5035 2 0x00000000 2014-01-07 18:54:31 ffff880831d23e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff88083264b058 ffff880831d23fd8 000000000000fb88 ffff88083264b058 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/11 S 000000000000000b 0 5036 2 0x00000000 2014-01-07 18:54:31 ffff88083266de30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff880831d6daf8 ffff88083266dfd8 000000000000fb88 ffff880831d6daf8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/12 S 000000000000000c 0 5037 2 0x00000000 2014-01-07 18:54:31 ffff88083266fe30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff880832561098 ffff88083266ffd8 000000000000fb88 ffff880832561098 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/13 S 000000000000000d 0 5038 2 0x00000000 2014-01-07 18:54:31 ffff8808151e1e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff8808325145f8 ffff8808151e1fd8 000000000000fb88 ffff8808325145f8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/14 S 000000000000000e 0 5039 2 0x00000000 2014-01-07 18:54:31 ffff8808151e3e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff880832515058 ffff8808151e3fd8 000000000000fb88 ffff880832515058 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 scsi_tgtd/15 S 000000000000000f 0 5040 2 0x00000000 2014-01-07 18:54:31 ffff880815241e30 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:31 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:31 ffff880832515ab8 ffff880815241fd8 000000000000fb88 ffff880832515ab8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:31 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:31 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 ib_fmr(mlx4_0 S 0000000000000009 0 5042 2 0x00000000 2014-01-07 18:54:31 ffff8808152a1e90 0000000000000046 0000000000000000 ffff88085c436740 2014-01-07 18:54:31 0000000000000009 ffff88085c436740 0000000000000009 0000000000000001 2014-01-07 18:54:31 ffff88083239e638 ffff8808152a1fd8 000000000000fb88 ffff88083239e638 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ib_fmr_cleanup_thread+0x7a/0xe0 [ib_core] 2014-01-07 18:54:31 [] ? ib_fmr_cleanup_thread+0x0/0xe0 [ib_core] 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 ib_fmr(mlx4_1 S 0000000000000009 0 5043 2 0x00000000 2014-01-07 18:54:31 ffff88080f2c1e90 0000000000000046 0000000000000000 ffff88085c436740 2014-01-07 18:54:31 0000000000000009 ffff88085c436740 0000000000000009 0000000000000001 2014-01-07 18:54:31 ffff880831e11058 ffff88080f2c1fd8 000000000000fb88 ffff880831e11058 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ib_fmr_cleanup_thread+0x7a/0xe0 [ib_core] 2014-01-07 18:54:31 [] ? ib_fmr_cleanup_thread+0x0/0xe0 [ib_core] 2014-01-07 18:54:31 [] kthread+0x96/0xa0 2014-01-07 18:54:31 [] child_rip+0xa/0x20 2014-01-07 18:54:31 [] ? kthread+0x0/0xa0 2014-01-07 18:54:31 [] ? child_rip+0x0/0x20 2014-01-07 18:54:31 opensm S 0000000000000000 0 5079 1 0x00000000 2014-01-07 18:54:31 ffff8808334f1e68 0000000000000086 0000000000000000 ffff8808334f1e2c 2014-01-07 18:54:31 ffff880800000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:31 ffff880833c025f8 ffff8808334f1fd8 000000000000fb88 ffff880833c025f8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? common_interrupt+0xe/0x13 2014-01-07 18:54:31 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:31 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:31 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:31 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:31 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000008 0 5081 1 0x00000000 2014-01-07 18:54:31 ffff88081528fbf8 0000000000000086 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:31 ffffffff00000008 000000013fe82000 ffff8800446f6740 0000000000000207 2014-01-07 18:54:31 ffff880831abfab8 ffff88081528ffd8 000000000000fb88 ffff880831abfab8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:31 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:31 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:31 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:31 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:31 [] do_futex+0x121/0xb60 2014-01-07 18:54:31 [] sys_futex+0x7b/0x170 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000003 0 5082 1 0x00000000 2014-01-07 18:54:31 ffff880828f1fbf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:31 01ff880840010dc0 ffffffff00000000 0000000000016740 ffff8800446101f0 2014-01-07 18:54:31 ffff880831039ab8 ffff880828f1ffd8 000000000000fb88 ffff880831039ab8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:31 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:31 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:31 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:31 [] do_futex+0x121/0xb60 2014-01-07 18:54:31 [] sys_futex+0x7b/0x170 2014-01-07 18:54:31 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:31 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000002 0 5083 1 0x00000000 2014-01-07 18:54:31 ffff880832551bf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:31 01ff880000019e40 ffffffff00000000 0000000000016740 ffff8800446901f0 2014-01-07 18:54:31 ffff88082710f058 ffff880832551fd8 000000000000fb88 ffff88082710f058 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:31 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:31 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:31 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:31 [] do_futex+0x121/0xb60 2014-01-07 18:54:31 [] sys_futex+0x7b/0x170 2014-01-07 18:54:31 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:31 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000008 0 5084 1 0x00000000 2014-01-07 18:54:31 ffff88083141bbf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:31 01ff880840010dc0 ffff880800000000 ffff880044656740 0000000000000400 2014-01-07 18:54:31 ffff880833c45ab8 ffff88083141bfd8 000000000000fb88 ffff880833c45ab8 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:31 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:31 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:31 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:31 [] do_futex+0x121/0xb60 2014-01-07 18:54:31 [] sys_futex+0x7b/0x170 2014-01-07 18:54:31 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:31 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000008 0 5085 1 0x00000000 2014-01-07 18:54:31 ffff8808151dfbf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:31 01ff880840010dc0 0000000000000000 ffff8808151dfba8 ffffffff81065a05 2014-01-07 18:54:31 ffff8808319d3098 ffff8808151dffd8 000000000000fb88 ffff8808319d3098 2014-01-07 18:54:31 Call Trace: 2014-01-07 18:54:31 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:31 [] ? enqueue_entity+0x125/0x410 2014-01-07 18:54:31 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:31 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:31 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:31 [] do_futex+0x121/0xb60 2014-01-07 18:54:31 [] sys_futex+0x7b/0x170 2014-01-07 18:54:31 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:31 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:31 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:31 opensm S 0000000000000000 0 5086 1 0x00000000 2014-01-07 18:54:32 ffff88083265dbf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:32 ffff88083265db78 ffffffff81277ab9 0000000000016740 ffff88085c4501f0 2014-01-07 18:54:32 ffff8808319d2638 ffff88083265dfd8 000000000000fb88 ffff8808319d2638 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:32 [] ? cpumask_next_and+0x29/0x50 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 000000000000000a 0 5087 1 0x00000000 2014-01-07 18:54:32 ffff88083251bbf8 0000000000000086 0000000000000000 ffff88083251bbbc 2014-01-07 18:54:32 ffff880800000000 ffff88083fe82400 ffff88085c4d6740 0000000000000207 2014-01-07 18:54:32 ffff88082885b098 ffff88083251bfd8 000000000000fb88 ffff88082885b098 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 000000000000000a 0 5088 1 0x00000000 2014-01-07 18:54:32 ffff8808319c9bf8 0000000000000086 0000000000000000 ffff8808289ce9f0 2014-01-07 18:54:32 0000000000000008 ffff8808289cea78 000000000000000a 0000000000000000 2014-01-07 18:54:32 ffff88082885a638 ffff8808319c9fd8 000000000000fb88 ffff88082885a638 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000008 0 5089 1 0x00000000 2014-01-07 18:54:32 ffff880832395bf8 0000000000000086 0000000000000000 ffff880832395bbc 2014-01-07 18:54:32 0000000000000008 ffff88083fe82000 ffff88085c476740 0000000000000400 2014-01-07 18:54:32 ffff8808335fd058 ffff880832395fd8 000000000000fb88 ffff8808335fd058 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000002 0 5090 1 0x00000000 2014-01-07 18:54:32 ffff88081507fbf8 0000000000000086 0000000000000000 000000008112bc13 2014-01-07 18:54:32 0000000000009e40 0000000000000046 00000000344f8650 ffff8808289cea30 2014-01-07 18:54:32 ffff8808335fc5f8 ffff88081507ffd8 000000000000fb88 ffff8808335fc5f8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 000000000000000a 0 5091 1 0x00000000 2014-01-07 18:54:32 ffff880832743bf8 0000000000000086 0000000000000000 ffff8808289ce9f0 2014-01-07 18:54:32 0000000000000008 ffff8808289cea78 0000000000000006 0000000000000000 2014-01-07 18:54:32 ffff880833c1b058 ffff880832743fd8 000000000000fb88 ffff880833c1b058 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000003 0 5092 1 0x00000000 2014-01-07 18:54:32 ffff8808151a5bf8 0000000000000086 0000000000000000 ffff8808151a5bbc 2014-01-07 18:54:32 0000000000000008 ffff88083fe82200 ffff8800446b6740 00000000000005fe 2014-01-07 18:54:32 ffff880833c1bab8 ffff8808151a5fd8 000000000000fb88 ffff880833c1bab8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 000000000000000a 0 5093 1 0x00000000 2014-01-07 18:54:32 ffff880832549bf8 0000000000000086 0000000000000000 ffff8808289ce9f0 2014-01-07 18:54:32 0000000000000008 ffff8808289cea78 000000000000000d 0000000000000000 2014-01-07 18:54:32 ffff880831253098 ffff880832549fd8 000000000000fb88 ffff880831253098 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000009 0 5094 1 0x00000000 2014-01-07 18:54:32 ffff880815093bf8 0000000000000086 0000000000000000 000000008112bc13 2014-01-07 18:54:32 0000000000000dc0 0000000000000046 0000000015093ba8 ffff8808289cea30 2014-01-07 18:54:32 ffff8808310c1ab8 ffff880815093fd8 000000000000fb88 ffff8808310c1ab8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000009 0 5095 1 0x00000000 2014-01-07 18:54:32 ffff880827bfdbf8 0000000000000086 0000000000000000 ffff8808289ce9f0 2014-01-07 18:54:32 0000000000000008 ffff8808289cea78 0000000000000012 0000000000000000 2014-01-07 18:54:32 ffff8808283abaf8 ffff880827bfdfd8 000000000000fb88 ffff8808283abaf8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000002 0 5096 1 0x00000000 2014-01-07 18:54:32 ffff880831d91bf8 0000000000000086 0000000000000000 ffff880831d91bbc 2014-01-07 18:54:32 0000000000000000 ffff88083fe81200 ffff88085c476740 0000000000000400 2014-01-07 18:54:32 ffff880832493058 ffff880831d91fd8 000000000000fb88 ffff880832493058 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000008 0 5097 1 0x00000000 2014-01-07 18:54:32 ffff880828f21bf8 0000000000000086 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:32 01ff880840010dc0 ffffffff00000000 0000000000016740 ffff8800446f01f0 2014-01-07 18:54:32 ffff8808324925f8 ffff880828f21fd8 000000000000fb88 ffff8808324925f8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000000 0 5098 1 0x00000000 2014-01-07 18:54:32 ffff880827017bf8 0000000000000086 0000000000000000 ffff880828a88df0 2014-01-07 18:54:32 0000000000000008 ffff880828a88e78 0000000000000031 0000000000000000 2014-01-07 18:54:32 ffff880833cfdaf8 ffff880827017fd8 000000000000fb88 ffff880833cfdaf8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] ? ib_umad_write+0x5d4/0x620 [ib_umad] 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000000 0 5099 1 0x00000000 2014-01-07 18:54:32 ffff880832717bf8 0000000000000086 ffff880832717b98 ffff880828333000 2014-01-07 18:54:32 ffff880832717b88 ffff88082685a780 ffff880832717b88 ffff8808208185c0 2014-01-07 18:54:32 ffff880833cfd098 ffff880832717fd8 000000000000fb88 ffff880833cfd098 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] ? transfer_objects+0x5c/0x80 2014-01-07 18:54:32 [] ? call_rcu+0xe/0x10 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] ? sys_renameat+0xb9/0x240 2014-01-07 18:54:32 [] ? destroy_inode+0x2f/0x60 2014-01-07 18:54:32 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000002 0 5100 1 0x00000000 2014-01-07 18:54:32 ffff880827227bf8 0000000000000086 0000000000000000 0000000000000000 2014-01-07 18:54:32 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:32 ffff880833609af8 ffff880827227fd8 000000000000fb88 ffff880833609af8 2014-01-07 18:54:32 Call Trace: 2014-01-07 18:54:32 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:32 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:32 [] do_futex+0x121/0xb60 2014-01-07 18:54:32 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:32 [] sys_futex+0x7b/0x170 2014-01-07 18:54:32 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:32 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:32 opensm S 0000000000000001 0 5101 1 0x00000000 2014-01-07 18:54:32 ffff88083436da58 0000000000000086 0000000000000000 ffff88083436da1c 2014-01-07 18:54:32 0000000000000000 ffff88083fe81000 ffff88085c496740 0000000000000400 2014-01-07 18:54:33 ffff880831e105f8 ffff88083436dfd8 000000000000fb88 ffff880831e105f8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:33 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:33 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:33 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:33 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:33 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:33 [] ? pollwake+0x0/0x60 2014-01-07 18:54:33 [] ? enqueue_entity+0x125/0x410 2014-01-07 18:54:33 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:33 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:33 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] ? wake_up_state+0x10/0x20 2014-01-07 18:54:33 [] ? wake_futex+0x40/0x60 2014-01-07 18:54:33 [] ? do_futex+0x6b2/0xb60 2014-01-07 18:54:33 [] ? sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] sys_poll+0x71/0x100 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000002 0 5118 1 0x00000000 2014-01-07 18:54:33 ffff880831435e68 0000000000000082 0000000000000000 ffff880831435e2c 2014-01-07 18:54:33 ffff880800000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:33 ffff880833584638 ffff880831435fd8 000000000000fb88 ffff880833584638 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:33 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:33 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:33 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:33 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000001 0 5120 1 0x00000000 2014-01-07 18:54:33 ffff88103194bbf8 0000000000000082 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:33 0000000000000001 000000013fe81200 ffff880044696740 00000000000002af 2014-01-07 18:54:33 ffff881031caa5f8 ffff88103194bfd8 000000000000fb88 ffff881031caa5f8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:33 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000000 0 5121 1 0x00000000 2014-01-07 18:54:33 ffff881030353bf8 0000000000000082 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:33 01ff880000019e40 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:33 ffff881031cab058 ffff881030353fd8 000000000000fb88 ffff881031cab058 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:33 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000008 0 5122 1 0x00000000 2014-01-07 18:54:33 ffff881032249bf8 0000000000000082 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:33 01ff880840010dc0 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:33 ffff88103006b058 ffff881032249fd8 000000000000fb88 ffff88103006b058 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:33 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000005 0 5123 1 0x00000000 2014-01-07 18:54:33 ffff881032361bf8 0000000000000082 0000000000000000 ffffffff8112bc13 2014-01-07 18:54:33 01ff880000019e40 ffffffff00000000 0000000000016740 ffff8800446901f0 2014-01-07 18:54:33 ffff881030bbbab8 ffff881032361fd8 000000000000fb88 ffff881030bbbab8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] ? __alloc_pages_nodemask+0x113/0x8d0 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:33 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000008 0 5124 1 0x00000000 2014-01-07 18:54:33 ffff88103039fbf8 0000000000000082 0000000000000000 ffff88103fc09940 2014-01-07 18:54:33 01ff88103039fbc8 ffffffff811675ab 0000000000000000 0000000100000000 2014-01-07 18:54:33 ffff881030bba5f8 ffff88103039ffd8 000000000000fb88 ffff881030bba5f8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] ? cache_alloc_refill+0x15b/0x240 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? math_state_restore+0x45/0x60 2014-01-07 18:54:33 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 000000000000000b 0 5125 1 0x00000000 2014-01-07 18:54:33 ffff881032a59bf8 0000000000000082 0000000000000000 ffff8808342f2df0 2014-01-07 18:54:33 0000000000000008 ffff8808342f2e78 0000000000000001 0000000000000000 2014-01-07 18:54:33 ffff8810341d5af8 ffff881032a59fd8 000000000000fb88 ffff8810341d5af8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000001 0 5126 1 0x00000000 2014-01-07 18:54:33 ffff8810331cdbf8 0000000000000082 0000000000000000 ffff8810331cdbbc 2014-01-07 18:54:33 ffff8810331cc000 ffff88083fe81200 ffff880044656740 00000000000002af 2014-01-07 18:54:33 ffff881030bc7af8 ffff8810331cdfd8 000000000000fb88 ffff881030bc7af8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000008 0 5127 1 0x00000000 2014-01-07 18:54:33 ffff881033241bf8 0000000000000082 0000000000000000 ffff881033241bbc 2014-01-07 18:54:33 01ff880800000000 ffff88083fe82000 ffff880044696740 00000000000005fe 2014-01-07 18:54:33 ffff881030bc6638 ffff881033241fd8 000000000000fb88 ffff881030bc6638 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000002 0 5128 1 0x00000000 2014-01-07 18:54:33 ffff881031cb9bf8 0000000000000082 0000000000000000 ffff8808342f2df0 2014-01-07 18:54:33 0000000000000008 ffff8808342f2e78 0000000000000012 0000000000000000 2014-01-07 18:54:33 ffff88103236fab8 ffff881031cb9fd8 000000000000fb88 ffff88103236fab8 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 000000000000000b 0 5129 1 0x00000000 2014-01-07 18:54:33 ffff881030a1dbf8 0000000000000082 0000000000000000 ffff881030a1dbbc 2014-01-07 18:54:33 0000000000000008 ffff88083fe82600 ffff88085c416740 0000000000000400 2014-01-07 18:54:33 ffff88103236f058 ffff881030a1dfd8 000000000000fb88 ffff88103236f058 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000000 0 5130 1 0x00000000 2014-01-07 18:54:33 ffff8810323dfbf8 0000000000000082 0000000000000000 000000008112bc13 2014-01-07 18:54:33 0000000000009e40 0000000000000046 00000000344f8f50 ffff8808342f2e30 2014-01-07 18:54:33 ffff8810319f3098 ffff8810323dffd8 000000000000fb88 ffff8810319f3098 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 0000000000000008 0 5131 1 0x00000000 2014-01-07 18:54:33 ffff88102f405bf8 0000000000000082 0000000000000000 ffff8808342f2df0 2014-01-07 18:54:33 0000000000000008 ffff8808342f2e78 0000000000000006 0000000000000000 2014-01-07 18:54:33 ffff8810319f2638 ffff88102f405fd8 000000000000fb88 ffff8810319f2638 2014-01-07 18:54:33 Call Trace: 2014-01-07 18:54:33 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:33 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:33 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:33 [] do_futex+0x121/0xb60 2014-01-07 18:54:33 [] sys_futex+0x7b/0x170 2014-01-07 18:54:33 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:33 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:33 opensm S 000000000000000b 0 5132 1 0x00000000 2014-01-07 18:54:33 ffff88103037bbf8 0000000000000082 0000000000000000 ffff88103037bbbc 2014-01-07 18:54:33 0000000000000008 ffff88083fe82600 ffff88085c416740 0000000000000400 2014-01-07 18:54:33 ffff881030a065f8 ffff88103037bfd8 000000000000fb88 ffff881030a065f8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 0000000000000002 0 5133 1 0x00000000 2014-01-07 18:54:34 ffff881031bd5bf8 0000000000000082 0000000000000000 ffff881031bd5bbc 2014-01-07 18:54:34 0000000000000000 ffff88083fe81400 ffff880044636740 0000000000000400 2014-01-07 18:54:34 ffff881030a07ab8 ffff881031bd5fd8 000000000000fb88 ffff881030a07ab8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 000000000000000b 0 5134 1 0x00000000 2014-01-07 18:54:34 ffff88103420dbf8 0000000000000082 0000000000000000 ffff88103420dbbc 2014-01-07 18:54:34 0000000000000dc0 ffff88083fe82600 ffff88085c416740 0000000000000207 2014-01-07 18:54:34 ffff8810308545f8 ffff88103420dfd8 000000000000fb88 ffff8810308545f8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 000000000000000b 0 5135 1 0x00000000 2014-01-07 18:54:34 ffff8810303f1bf8 0000000000000082 0000000000000000 ffff8810303f1bbc 2014-01-07 18:54:34 0000000000000008 ffff88083fe82600 ffff88085c416740 000000000000af5c 2014-01-07 18:54:34 ffff881030855058 ffff8810303f1fd8 000000000000fb88 ffff881030855058 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 0000000000000002 0 5136 1 0x00000000 2014-01-07 18:54:34 ffff881030fc5bf8 0000000000000082 0000000000000000 ffff881030fc5bbc 2014-01-07 18:54:34 01ff880000019e40 ffff88083fe81400 ffff880044616740 00000000000002af 2014-01-07 18:54:34 ffff881030855ab8 ffff881030fc5fd8 000000000000fb88 ffff881030855ab8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 0000000000000006 0 5137 1 0x00000000 2014-01-07 18:54:34 ffff8810303a3bf8 0000000000000082 0000000000000000 ffff8808342ec1f0 2014-01-07 18:54:34 0000000000000008 ffff8808342ec278 0000000000000031 0000000000000000 2014-01-07 18:54:34 ffff8810302ed058 ffff8810303a3fd8 000000000000fb88 ffff8810302ed058 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? futex_wake+0x10e/0x120 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? ib_umad_write+0x5d4/0x620 [ib_umad] 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? sys_write+0x7c/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 0000000000000000 0 5138 1 0x00000000 2014-01-07 18:54:34 ffff881030951bf8 0000000000000082 0000000000000000 ffff880828333000 2014-01-07 18:54:34 ffff881030951b88 ffff88082685a780 ffff881030951b88 ffff8808208185c0 2014-01-07 18:54:34 ffff8810302edab8 ffff881030951fd8 000000000000fb88 ffff8810302edab8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? security_inode_permission+0x1f/0x30 2014-01-07 18:54:34 [] ? call_rcu+0xe/0x10 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? sys_renameat+0xb9/0x240 2014-01-07 18:54:34 [] ? destroy_inode+0x2f/0x60 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 000000000000000b 0 5139 1 0x00000000 2014-01-07 18:54:34 ffff881031901bf8 0000000000000082 0000000000000000 0000000000000000 2014-01-07 18:54:34 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:34 ffff8810302ec5f8 ffff881031901fd8 000000000000fb88 ffff8810302ec5f8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 opensm S 0000000000000002 0 5140 1 0x00000000 2014-01-07 18:54:34 ffff8808323e7a58 0000000000000082 0000000000000000 ffff8808323e7a1c 2014-01-07 18:54:34 ffff880000000000 ffff88083fe81000 ffff880044636740 0000000000000400 2014-01-07 18:54:34 ffff880833503af8 ffff8808323e7fd8 000000000000fb88 ffff880833503af8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:34 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:34 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:34 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:34 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:34 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:34 [] ? pollwake+0x0/0x60 2014-01-07 18:54:34 [] ? enqueue_entity+0x125/0x410 2014-01-07 18:54:34 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:34 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:34 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:34 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:34 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:34 [] ? wake_up_state+0x10/0x20 2014-01-07 18:54:34 [] ? wake_futex+0x40/0x60 2014-01-07 18:54:34 [] ? do_futex+0x6b2/0xb60 2014-01-07 18:54:34 [] ? sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] sys_poll+0x71/0x100 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 0000000000000005 0 5149 1 0x00000000 2014-01-07 18:54:34 ffff88083142dbf8 0000000000000082 0000000000000000 0000000600001000 2014-01-07 18:54:34 ffff880832ec6000 ffff8808334880cf ffff88080e11c138 ffff8808334880d0 2014-01-07 18:54:34 ffff880831d6c638 ffff88083142dfd8 000000000000fb88 ffff880831d6c638 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? sysfs_create_file+0x26/0x30 2014-01-07 18:54:34 [] ? scsi_scan_target+0xde/0xf0 2014-01-07 18:54:34 [] ? srp_create_target+0x332/0xe60 [ib_srp] 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? fput+0x25/0x30 2014-01-07 18:54:34 [] ? filp_close+0x5d/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 000000000000000a 0 5157 1 0x00000000 2014-01-07 18:54:34 ffff880815229e38 0000000000000082 0000000000000000 0000000000000000 2014-01-07 18:54:34 0000000000000000 0000000000000000 0000000000000000 ffffea0038b648a0 2014-01-07 18:54:34 ffff88082830faf8 ffff880815229fd8 000000000000fb88 ffff88082830faf8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] ib_uverbs_event_read+0x11d/0x240 [ib_uverbs] 2014-01-07 18:54:34 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:34 [] ? security_file_permission+0x16/0x20 2014-01-07 18:54:34 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:34 [] sys_read+0x51/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 0000000000000005 0 5158 1 0x00000000 2014-01-07 18:54:34 ffff8808271c9e38 0000000000000082 0000000000000000 0000000000000000 2014-01-07 18:54:34 0000000000000000 0000000000000000 0000000000000000 ffffea0038af5b70 2014-01-07 18:54:34 ffff88083274a638 ffff8808271c9fd8 000000000000fb88 ffff88083274a638 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:34 [] ib_uverbs_event_read+0x11d/0x240 [ib_uverbs] 2014-01-07 18:54:34 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:34 [] ? security_file_permission+0x16/0x20 2014-01-07 18:54:34 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:34 [] sys_read+0x51/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 000000000000000b 0 5159 1 0x00000000 2014-01-07 18:54:34 ffff880832b75bf8 0000000000000082 0000000000000000 0000000000000000 2014-01-07 18:54:34 ffff880832b75b78 ffffffff81277ab9 0000000000016740 ffff8800446101f0 2014-01-07 18:54:34 ffff8808328c7058 ffff880832b75fd8 000000000000fb88 ffff8808328c7058 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] ? cpumask_next_and+0x29/0x50 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? perf_event_task_sched_out+0x33/0x80 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:34 [] ? __free_pages+0x60/0xa0 2014-01-07 18:54:34 [] ? free_pages+0x49/0x50 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 0000000000000005 0 5153 1 0x00000000 2014-01-07 18:54:34 ffff88103085bbf8 0000000000000086 0000000000000000 0000000600001000 2014-01-07 18:54:34 ffff880831068400 ffff8808334880cf ffff880827850138 ffff8808334880d0 2014-01-07 18:54:34 ffff881031cabab8 ffff88103085bfd8 000000000000fb88 ffff881031cabab8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:34 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:34 [] ? sysfs_create_file+0x26/0x30 2014-01-07 18:54:34 [] ? scsi_scan_target+0xde/0xf0 2014-01-07 18:54:34 [] ? srp_create_target+0x332/0xe60 [ib_srp] 2014-01-07 18:54:34 [] do_futex+0x121/0xb60 2014-01-07 18:54:34 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:54:34 [] sys_futex+0x7b/0x170 2014-01-07 18:54:34 [] ? fput+0x25/0x30 2014-01-07 18:54:34 [] ? filp_close+0x5d/0x90 2014-01-07 18:54:34 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:34 srp_daemon S 0000000000000009 0 5160 1 0x00000000 2014-01-07 18:54:34 ffff8808150bfe38 0000000000000086 0000000000000000 ffff8808328d9540 2014-01-07 18:54:34 ffff880044696740 0000000000000001 00000000ffffffff 00000000ffffffff 2014-01-07 18:54:34 ffff8808328c7ab8 ffff8808150bffd8 000000000000fb88 ffff8808328c7ab8 2014-01-07 18:54:34 Call Trace: 2014-01-07 18:54:34 [] ib_uverbs_event_read+0x11d/0x240 [ib_uverbs] 2014-01-07 18:54:34 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:34 [] ? security_file_permission+0x16/0x20 2014-01-07 18:54:34 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:35 [] sys_read+0x51/0x90 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 srp_daemon S 000000000000000a 0 5161 1 0x00000000 2014-01-07 18:54:35 ffff8808320c3e38 0000000000000086 0000000000000000 ffff880827311500 2014-01-07 18:54:35 ffff880044696740 0000000000000001 00000000ffffffff 00000000ffffffff 2014-01-07 18:54:35 ffff880831071098 ffff8808320c3fd8 000000000000fb88 ffff880831071098 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ib_uverbs_event_read+0x11d/0x240 [ib_uverbs] 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? security_file_permission+0x16/0x20 2014-01-07 18:54:35 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:35 [] sys_read+0x51/0x90 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 srp_daemon S 0000000000000006 0 5162 1 0x00000000 2014-01-07 18:54:35 ffff8808151ffbf8 0000000000000086 0000000000000000 0000000000000000 2014-01-07 18:54:35 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:35 ffff88083264bab8 ffff8808151fffd8 000000000000fb88 ffff88083264bab8 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] futex_wait_queue_me+0xb9/0xf0 2014-01-07 18:54:35 [] futex_wait+0x1f8/0x380 2014-01-07 18:54:35 [] do_futex+0x121/0xb60 2014-01-07 18:54:35 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:35 [] ? __free_pages+0x60/0xa0 2014-01-07 18:54:35 [] ? free_pages+0x49/0x50 2014-01-07 18:54:35 [] sys_futex+0x7b/0x170 2014-01-07 18:54:35 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 sshd S 0000000000000009 0 5178 1 0x00000000 2014-01-07 18:54:35 ffff88082707d908 0000000000000082 0000000000000000 ffff880840010dc0 2014-01-07 18:54:35 0000000000000002 0000000000000002 0000000000000003 ffff880840021b40 2014-01-07 18:54:35 ffff880833e8b098 ffff88082707dfd8 000000000000fb88 ffff880833e8b098 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:35 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:35 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:35 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:35 [] do_select+0x57c/0x6c0 2014-01-07 18:54:35 [] ? alloc_pages_vma+0x9a/0x150 2014-01-07 18:54:35 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? find_get_page+0x1e/0xa0 2014-01-07 18:54:35 [] ? filemap_fault+0xd3/0x500 2014-01-07 18:54:35 [] ? unlock_page+0x27/0x30 2014-01-07 18:54:35 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:35 [] ? handle_pte_fault+0xf7/0xb50 2014-01-07 18:54:35 [] ? __pagevec_free+0x44/0x90 2014-01-07 18:54:35 [] ? page_remove_rmap+0x54/0xa0 2014-01-07 18:54:35 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:54:35 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:35 [] ? cpumask_any_but+0x31/0x50 2014-01-07 18:54:35 [] ? unmap_region+0x110/0x130 2014-01-07 18:54:35 [] sys_select+0x47/0x110 2014-01-07 18:54:35 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 xinetd S 0000000000000002 0 5188 1 0x00000000 2014-01-07 18:54:35 ffff880831aefa58 0000000000000086 0000000000000000 ffffffff81065a05 2014-01-07 18:54:35 ffff880831aef9c8 ffff8810341d4b18 ffff8800446167a8 ffff880044616740 2014-01-07 18:54:35 ffff880831a45ab8 ffff880831aeffd8 000000000000fb88 ffff880831a45ab8 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ? enqueue_entity+0x125/0x410 2014-01-07 18:54:35 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:35 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:35 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:35 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:35 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:35 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? proc_flush_task+0x15a/0x2b0 2014-01-07 18:54:35 [] ? call_rcu+0xe/0x10 2014-01-07 18:54:35 [] ? release_task+0x33f/0x4b0 2014-01-07 18:54:35 [] ? remove_wait_queue+0x3c/0x50 2014-01-07 18:54:35 [] ? do_wait+0x17f/0x240 2014-01-07 18:54:35 [] ? sys_wait4+0xae/0x100 2014-01-07 18:54:35 [] sys_poll+0x71/0x100 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 ntpd S 0000000000000005 0 5207 1 0x00000000 2014-01-07 18:54:35 ffff880834287908 0000000000000082 0000000000000000 ffff8808342878cc 2014-01-07 18:54:35 0000001000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:54:35 ffff880828f25ab8 ffff880834287fd8 000000000000fb88 ffff880828f25ab8 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:35 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:35 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:35 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:35 [] do_select+0x57c/0x6c0 2014-01-07 18:54:35 [] ? put_cmsg+0xc0/0x100 2014-01-07 18:54:35 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] ? pollwake+0x0/0x60 2014-01-07 18:54:35 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:54:35 [] ? __sys_recvmsg+0x13c/0x2f0 2014-01-07 18:54:35 [] ? move_addr_to_user+0x93/0xb0 2014-01-07 18:54:35 [] ? sys_recvfrom+0xee/0x180 2014-01-07 18:54:35 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:35 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:35 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:35 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:35 [] sys_select+0x47/0x110 2014-01-07 18:54:35 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:35 cfs_rh_00 S 000000000000000a 0 5217 2 0x00000000 2014-01-07 18:54:35 ffff880828f3fe40 0000000000000046 0000000000000000 ffffffff81057310 2014-01-07 18:54:35 ffff880828f3fdb0 ffffffff81277b29 ffff880828f3fdd0 ffffffff81277b6a 2014-01-07 18:54:35 ffff880828f25058 ffff880828f3ffd8 000000000000fb88 ffff880828f25058 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:35 [] ? __next_cpu+0x19/0x30 2014-01-07 18:54:35 [] ? __any_online_cpu+0x2a/0x50 2014-01-07 18:54:35 [] cfs_wi_scheduler+0x3d7/0x460 [libcfs] 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? cfs_wi_scheduler+0x0/0x460 [libcfs] 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:35 cfs_rh_01 S 0000000000000009 0 5218 2 0x00000000 2014-01-07 18:54:35 ffff8808271a3e40 0000000000000046 ffff8808271a3e08 ffff8808271a3e04 2014-01-07 18:54:35 ffff8808271a3db0 ffff88083fe82200 ffff880044656740 0000000000000bf6 2014-01-07 18:54:35 ffff880833e8a638 ffff8808271a3fd8 000000000000fb88 ffff880833e8a638 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] cfs_wi_scheduler+0x3d7/0x460 [libcfs] 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? cfs_wi_scheduler+0x0/0x460 [libcfs] 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:35 cfs_rh_02 S 0000000000000009 0 5219 2 0x00000000 2014-01-07 18:54:35 ffff88083289be40 0000000000000046 0000000000000000 ffffffff81057310 2014-01-07 18:54:35 ffff88083289bdb0 ffffffff81277b29 ffff88083289bdd0 ffffffff81277b6a 2014-01-07 18:54:35 ffff8808342b0638 ffff88083289bfd8 000000000000fb88 ffff8808342b0638 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:35 [] ? __next_cpu+0x19/0x30 2014-01-07 18:54:35 [] ? __any_online_cpu+0x2a/0x50 2014-01-07 18:54:35 [] cfs_wi_scheduler+0x3d7/0x460 [libcfs] 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? cfs_wi_scheduler+0x0/0x460 [libcfs] 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:35 cfs_rh_03 S 0000000000000009 0 5220 2 0x00000000 2014-01-07 18:54:35 ffff8808335efe40 0000000000000046 0000000000000000 ffffffff81057310 2014-01-07 18:54:35 ffff8808335efdb0 ffffffff81277b29 ffff8808335efdd0 ffffffff81277b6a 2014-01-07 18:54:35 ffff88083242b098 ffff8808335effd8 000000000000fb88 ffff88083242b098 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:35 [] ? __next_cpu+0x19/0x30 2014-01-07 18:54:35 [] ? __any_online_cpu+0x2a/0x50 2014-01-07 18:54:35 [] cfs_wi_scheduler+0x3d7/0x460 [libcfs] 2014-01-07 18:54:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:35 [] ? cfs_wi_scheduler+0x0/0x460 [libcfs] 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:35 scsi_eh_6 S 000000000000000a 0 5231 2 0x00000000 2014-01-07 18:54:35 ffff880831ddde30 0000000000000046 0000000000000000 ffff88085c4567a8 2014-01-07 18:54:35 ffff880831ddddb0 ffffffff81057310 ffff880833c02ad8 ffff88085c4567a8 2014-01-07 18:54:35 ffff880833c03058 ffff880831dddfd8 000000000000fb88 ffff880833c03058 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:35 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:35 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:35 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:35 scsi_eh_7 S 0000000000000009 0 5232 2 0x00000000 2014-01-07 18:54:35 ffff8808150f5e30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:35 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:35 ffff880833cdb058 ffff8808150f5fd8 000000000000fb88 ffff880833cdb058 2014-01-07 18:54:35 Call Trace: 2014-01-07 18:54:35 [] scsi_error_handler+0x96/0x6b0 2014-01-07 18:54:35 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:35 [] ? scsi_error_handler+0x0/0x6b0 2014-01-07 18:54:35 [] kthread+0x96/0xa0 2014-01-07 18:54:35 [] child_rip+0xa/0x20 2014-01-07 18:54:35 [] ? kthread+0x0/0xa0 2014-01-07 18:54:35 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 udevd S 0000000000000001 0 5329 2171 0x00000000 2014-01-07 18:54:36 ffff88080e301a08 0000000000000082 ffff88080e3019d0 ffff88080e3019cc 2014-01-07 18:54:36 0000000000000001 ffff88083fe81200 ffff880044616740 00000000000006e2 2014-01-07 18:54:36 ffff880834305af8 ffff88080e301fd8 000000000000fb88 ffff880834305af8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:36 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:36 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:36 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:36 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:36 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:36 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:36 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:36 [] ? pollwake+0x0/0x60 2014-01-07 18:54:36 [] ? sock_def_readable+0x44/0x80 2014-01-07 18:54:36 [] ? unix_dgram_sendmsg+0x5e3/0x6b0 2014-01-07 18:54:36 [] ? sock_sendmsg+0x123/0x150 2014-01-07 18:54:36 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:36 [] ? __sys_sendmsg+0x406/0x420 2014-01-07 18:54:36 [] ? sock_destroy_inode+0x19/0x20 2014-01-07 18:54:36 [] ? sys_sendto+0x139/0x190 2014-01-07 18:54:36 [] sys_ppoll+0x4c/0x180 2014-01-07 18:54:36 [] ? sys_sendmsg+0x49/0x90 2014-01-07 18:54:36 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:36 udevd S 0000000000000003 0 5335 2171 0x00000000 2014-01-07 18:54:36 ffff881030b5da08 0000000000000082 0000000000000000 0000000000000000 2014-01-07 18:54:36 0000000000000003 0000000000000000 0000000000000000 ffff8800446167a8 2014-01-07 18:54:36 ffff88102f4d7058 ffff881030b5dfd8 000000000000fb88 ffff88102f4d7058 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:36 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:36 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:36 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:54:36 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:36 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:36 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:36 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:36 [] ? pollwake+0x0/0x60 2014-01-07 18:54:36 [] ? sock_def_readable+0x44/0x80 2014-01-07 18:54:36 [] ? unix_dgram_sendmsg+0x5e3/0x6b0 2014-01-07 18:54:36 [] ? sock_sendmsg+0x123/0x150 2014-01-07 18:54:36 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:36 [] ? __sys_sendmsg+0x406/0x420 2014-01-07 18:54:36 [] ? sock_destroy_inode+0x19/0x20 2014-01-07 18:54:36 [] ? sys_sendto+0x139/0x190 2014-01-07 18:54:36 [] sys_ppoll+0x4c/0x180 2014-01-07 18:54:36 [] ? sys_sendmsg+0x49/0x90 2014-01-07 18:54:36 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:36 kdmflush S 000000000000000a 0 5383 2 0x00000000 2014-01-07 18:54:36 ffff88103000fe30 0000000000000046 0000000000000000 0000000000000001 2014-01-07 18:54:36 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:36 ffff8810301f6638 ffff88103000ffd8 000000000000fb88 ffff8810301f6638 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:36 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:36 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:36 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_kmem_cach S 0000000000000000 0 5399 2 0x00000000 2014-01-07 18:54:36 ffff881030f1fe20 0000000000000046 ffff881030f1fdb0 ffff88007fac62c0 2014-01-07 18:54:36 ffff880832f558d0 ffff88080da00000 ffffc9003d3bc000 ffff88080da080a8 2014-01-07 18:54:36 ffff881031e91ab8 ffff881030f1ffd8 000000000000fb88 ffff881031e91ab8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000000 0 5400 2 0x00000000 2014-01-07 18:54:36 ffff881031c55e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:36 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:36 ffff8810300e3ab8 ffff881031c55fd8 000000000000fb88 ffff8810300e3ab8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000001 0 5401 2 0x00000000 2014-01-07 18:54:36 ffff8810301fbe20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:36 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:36 ffff8810309605f8 ffff8810301fbfd8 000000000000fb88 ffff8810309605f8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000002 0 5402 2 0x00000000 2014-01-07 18:54:36 ffff8810302e5e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:36 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:36 ffff881030961058 ffff8810302e5fd8 000000000000fb88 ffff881030961058 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000003 0 5403 2 0x00000000 2014-01-07 18:54:36 ffff881031b85e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:36 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:36 ffff881031b8faf8 ffff881031b85fd8 000000000000fb88 ffff881031b8faf8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000004 0 5404 2 0x00000000 2014-01-07 18:54:36 ffff881030ea9e20 0000000000000046 ffff881030ea9de8 ffff881030ea9de4 2014-01-07 18:54:36 0000000000000000 ffff88083fe81800 ffff8800446b6740 000000000000af5c 2014-01-07 18:54:36 ffff881030a7daf8 ffff881030ea9fd8 000000000000fb88 ffff881030a7daf8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000005 0 5405 2 0x00000000 2014-01-07 18:54:36 ffff881030e7be20 0000000000000046 ffff881030e7bd90 ffffffff81096db6 2014-01-07 18:54:36 0000000000000000 ffff880833e41f70 ffff881030e7bde0 ffffffff81051439 2014-01-07 18:54:36 ffff881030bbb058 ffff881030e7bfd8 000000000000fb88 ffff881030bbb058 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:36 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000006 0 5406 2 0x00000000 2014-01-07 18:54:36 ffff881032c5be20 0000000000000046 ffff881032c5bde8 ffff881032c5bde4 2014-01-07 18:54:36 0000000000000000 ffff88083fe81c00 ffff8800446b6740 000000000000af5c 2014-01-07 18:54:36 ffff88103006bab8 ffff881032c5bfd8 000000000000fb88 ffff88103006bab8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000007 0 5407 2 0x00000000 2014-01-07 18:54:36 ffff881030f5de20 0000000000000046 ffff881030f5dde8 ffff881030f5dde4 2014-01-07 18:54:36 0000000000000000 ffff88083fe81e00 ffff8800446b6740 000000000000af5c 2014-01-07 18:54:36 ffff8810319f3af8 ffff881030f5dfd8 000000000000fb88 ffff8810319f3af8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000008 0 5408 2 0x00000000 2014-01-07 18:54:36 ffff881033133e20 0000000000000046 ffff881033133de8 ffff881033133de4 2014-01-07 18:54:36 0000000000000000 ffff88083fe82000 ffff8800446b6740 000000000000e87b 2014-01-07 18:54:36 ffff881037d47098 ffff881033133fd8 000000000000fb88 ffff881037d47098 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:36 [] kthread+0x96/0xa0 2014-01-07 18:54:36 [] child_rip+0xa/0x20 2014-01-07 18:54:36 [] ? kthread+0x0/0xa0 2014-01-07 18:54:36 [] ? child_rip+0x0/0x20 2014-01-07 18:54:36 spl_system_ta S 0000000000000009 0 5409 2 0x00000000 2014-01-07 18:54:36 ffff8810332d3e20 0000000000000046 ffff8810332d3de8 ffff8810332d3de4 2014-01-07 18:54:36 0000000000000000 ffff88083fe82200 ffff8800446b6740 000000000000a97c 2014-01-07 18:54:36 ffff88102f03bab8 ffff8810332d3fd8 000000000000fb88 ffff88102f03bab8 2014-01-07 18:54:36 Call Trace: 2014-01-07 18:54:36 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:36 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000a 0 5410 2 0x00000000 2014-01-07 18:54:37 ffff881031d45e20 0000000000000046 ffff881031d45de8 ffff881031d45de4 2014-01-07 18:54:37 0000000000000000 ffff88083fe82400 ffff8800446b6740 000000000000ad7c 2014-01-07 18:54:37 ffff88102f03b058 ffff881031d45fd8 000000000000fb88 ffff88102f03b058 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000b 0 5411 2 0x00000000 2014-01-07 18:54:37 ffff88080dca5e20 0000000000000046 ffffffff81e26bc0 000000000000ae8f 2014-01-07 18:54:37 ffff88085c436740 000000000000ae8f 0000000000000001 000000000000002a 2014-01-07 18:54:37 ffff880831ce7098 ffff88080dca5fd8 000000000000fb88 ffff880831ce7098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000c 0 5412 2 0x00000000 2014-01-07 18:54:37 ffff88080dca7e20 0000000000000046 ffffffff81e26bc0 000000000000ae8f 2014-01-07 18:54:37 ffff88085c436740 000000000000ae8f 0000000000000001 000000000000002a 2014-01-07 18:54:37 ffff8808328d9098 ffff88080dca7fd8 000000000000fb88 ffff8808328d9098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000d 0 5413 2 0x00000000 2014-01-07 18:54:37 ffff88080dca9e20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:37 ffff880044656740 ffff880833e41f70 ffff88080dca9de0 ffffffff81051439 2014-01-07 18:54:37 ffff88080dc26638 ffff88080dca9fd8 000000000000fb88 ffff88080dc26638 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:37 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000e 0 5414 2 0x00000000 2014-01-07 18:54:37 ffff88080dcabe20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:37 ffff880044656740 ffff880833e41f70 ffff88080dcabde0 ffffffff81051439 2014-01-07 18:54:37 ffff88083113c5f8 ffff88080dcabfd8 000000000000fb88 ffff88083113c5f8 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:37 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 spl_system_ta S 000000000000000f 0 5415 2 0x00000000 2014-01-07 18:54:37 ffff88080dcade20 0000000000000046 ffff88080dcadd90 ffffffff81096db6 2014-01-07 18:54:37 ffff880044656740 ffff880833e41f70 ffff88080dcadde0 ffffffff8100bb8e 2014-01-07 18:54:37 ffff88082836b058 ffff88080dcadfd8 000000000000fb88 ffff88082836b058 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:37 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:37 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:37 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kmpath_rdacd S 0000000000000009 0 5438 2 0x00000000 2014-01-07 18:54:37 ffff88080e3f5e30 0000000000000046 0000000000000000 ffff88085c4367a8 2014-01-07 18:54:37 ffff88080e3f5db0 ffffffff81057310 ffff8808328d9578 ffff88085c4367a8 2014-01-07 18:54:37 ffff8808328d9af8 ffff88080e3f5fd8 000000000000fb88 ffff8808328d9af8 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kdmflush S 0000000000000002 0 5452 2 0x00000000 2014-01-07 18:54:37 ffff881030bfde30 0000000000000046 ffff881030bfddf8 ffff881030bfddf4 2014-01-07 18:54:37 ffff881030bfddb0 ffff88083fe81400 ffff880044616740 0000000000000519 2014-01-07 18:54:37 ffff88103031b098 ffff881030bfdfd8 000000000000fb88 ffff88103031b098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kdmflush S 0000000000000008 0 5457 2 0x00000000 2014-01-07 18:54:37 ffff88103001de30 0000000000000046 0000000000000000 ffff88103001ddf4 2014-01-07 18:54:37 0000000000000001 ffff88083fe82000 ffff880044636740 00000000000004f6 2014-01-07 18:54:37 ffff8810321cc5f8 ffff88103001dfd8 000000000000fb88 ffff8810321cc5f8 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kdmflush S 0000000000000008 0 5471 2 0x00000000 2014-01-07 18:54:37 ffff881031a61e30 0000000000000046 ffff88083108ab28 ffff8800446567a8 2014-01-07 18:54:37 ffff881031a61db0 ffffffff81057310 ffff88083108ab18 ffff8800446567a8 2014-01-07 18:54:37 ffff881032633af8 ffff881031a61fd8 000000000000fb88 ffff881032633af8 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:37 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:37 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kdmflush S 000000000000000b 0 5478 2 0x00000000 2014-01-07 18:54:37 ffff88103319be30 0000000000000046 0000000000000000 ffff8800446367a8 2014-01-07 18:54:37 0000000000000001 ffff880044636740 ffff88103319bde0 ffffffff8106669b 2014-01-07 18:54:37 ffff8810319665f8 ffff88103319bfd8 000000000000fb88 ffff8810319665f8 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? dequeue_task_fair+0x12b/0x130 2014-01-07 18:54:37 [] ? prepare_to_wait+0x4e/0x80 2014-01-07 18:54:37 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 kdmflush S 0000000000000001 0 5488 2 0x00000000 2014-01-07 18:54:37 ffff881032a53e30 0000000000000046 ffff881032a53df8 ffff881032a53df4 2014-01-07 18:54:37 0000000000000000 ffff88083fe81200 ffff88085c436740 0000000000000414 2014-01-07 18:54:37 ffff881030bc7098 ffff881032a53fd8 000000000000fb88 ffff881030bc7098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] ? dm_wq_work+0x0/0xf0 [dm_mod] 2014-01-07 18:54:37 [] worker_thread+0x1fc/0x2a0 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? worker_thread+0x0/0x2a0 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 arc_adapt S 0000000000000008 0 5505 2 0x00000000 2014-01-07 18:54:37 ffff881033287d70 0000000000000046 0000000000000000 ffff881033287d34 2014-01-07 18:54:37 ffff880e00000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:37 ffff881031b89098 ffff881033287fd8 000000000000fb88 ffff881031b89098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:37 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:37 [] __cv_timedwait_common+0x9f/0x130 [spl] 2014-01-07 18:54:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:37 [] ? arc_adapt_thread+0x0/0x160 [zfs] 2014-01-07 18:54:37 [] __cv_timedwait_interruptible+0x13/0x20 [spl] 2014-01-07 18:54:37 [] arc_adapt_thread+0x9c/0x160 [zfs] 2014-01-07 18:54:37 [] thread_generic_wrapper+0x5f/0x70 [spl] 2014-01-07 18:54:37 [] ? thread_generic_wrapper+0x0/0x70 [spl] 2014-01-07 18:54:37 [] kthread+0x96/0xa0 2014-01-07 18:54:37 [] child_rip+0xa/0x20 2014-01-07 18:54:37 [] ? kthread+0x0/0xa0 2014-01-07 18:54:37 [] ? child_rip+0x0/0x20 2014-01-07 18:54:37 l2arc_feed S 0000000000000009 0 5506 2 0x00000000 2014-01-07 18:54:37 ffff881032451c90 0000000000000046 0000000000000000 ffff881032451c54 2014-01-07 18:54:37 0000000000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:54:37 ffff8810302df098 ffff881032451fd8 000000000000fb88 ffff8810302df098 2014-01-07 18:54:37 Call Trace: 2014-01-07 18:54:37 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:37 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:37 [] __cv_timedwait_common+0x9f/0x130 [spl] 2014-01-07 18:54:38 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:38 [] __cv_timedwait_interruptible+0x13/0x20 [spl] 2014-01-07 18:54:38 [] l2arc_feed_thread+0x55/0xa40 [zfs] 2014-01-07 18:54:38 [] ? enqueue_entity+0x56/0x410 2014-01-07 18:54:38 [] ? enqueue_task_fair+0xfb/0x100 2014-01-07 18:54:38 [] ? l2arc_feed_thread+0x0/0xa40 [zfs] 2014-01-07 18:54:38 [] thread_generic_wrapper+0x5f/0x70 [spl] 2014-01-07 18:54:38 [] ? thread_generic_wrapper+0x0/0x70 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 z_unmount/0 S 0000000000000000 0 5507 2 0x00000000 2014-01-07 18:54:38 ffff881032455e20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:38 0000000000000000 ffff880832ee0c70 ffff881032455de0 ffffffff81051439 2014-01-07 18:54:38 ffff881031c14638 ffff881032455fd8 000000000000fb88 ffff881031c14638 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:38 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/0 S 0000000000000000 0 5508 2 0x00000000 2014-01-07 18:54:38 ffff881032457e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881031b88638 ffff881032457fd8 000000000000fb88 ffff881031b88638 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/1 S 0000000000000001 0 5509 2 0x00000000 2014-01-07 18:54:38 ffff881030f23e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f21ab8 ffff881030f23fd8 000000000000fb88 ffff881030f21ab8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/2 S 0000000000000002 0 5510 2 0x00000000 2014-01-07 18:54:38 ffff881030f27e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f21058 ffff881030f27fd8 000000000000fb88 ffff881030f21058 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/3 S 0000000000000003 0 5511 2 0x00000000 2014-01-07 18:54:38 ffff8810318a9e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f205f8 ffff8810318a9fd8 000000000000fb88 ffff881030f205f8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/4 S 0000000000000004 0 5512 2 0x00000000 2014-01-07 18:54:38 ffff8810318ade20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff8810318abaf8 ffff8810318adfd8 000000000000fb88 ffff8810318abaf8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/5 S 0000000000000005 0 5513 2 0x00000000 2014-01-07 18:54:38 ffff881030f69e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff8810318ab098 ffff881030f69fd8 000000000000fb88 ffff8810318ab098 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/6 S 0000000000000006 0 5514 2 0x00000000 2014-01-07 18:54:38 ffff881030f6be20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff8810318aa638 ffff881030f6bfd8 000000000000fb88 ffff8810318aa638 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/7 S 0000000000000007 0 5515 2 0x00000000 2014-01-07 18:54:38 ffff881030f6fe20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f6dab8 ffff881030f6ffd8 000000000000fb88 ffff881030f6dab8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/8 S 0000000000000008 0 5516 2 0x00000000 2014-01-07 18:54:38 ffff88103008be20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f6d058 ffff88103008bfd8 000000000000fb88 ffff881030f6d058 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/9 S 0000000000000009 0 5517 2 0x00000000 2014-01-07 18:54:38 ffff88103008de20 0000000000000046 0000000000000000 000000000000040f 2014-01-07 18:54:38 0000000000000002 0000000000000001 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff881030f6c5f8 ffff88103008dfd8 000000000000fb88 ffff881030f6c5f8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/10 S 000000000000000a 0 5518 2 0x00000000 2014-01-07 18:54:38 ffff8810333c1e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:38 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:38 ffff88103008faf8 ffff8810333c1fd8 000000000000fb88 ffff88103008faf8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/11 S 000000000000000b 0 5519 2 0x00000000 2014-01-07 18:54:38 ffff8810333c5e20 0000000000000046 ffff8810333c5de8 ffff8810333c5de4 2014-01-07 18:54:38 0000000000000000 ffff88083fe82600 ffff88085c496740 0000000000000207 2014-01-07 18:54:38 ffff88103008f098 ffff8810333c5fd8 000000000000fb88 ffff88103008f098 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/12 S 000000000000000c 0 5520 2 0x00000000 2014-01-07 18:54:38 ffff8810333c7e20 0000000000000046 ffff8810333c7d90 ffffffff81096db6 2014-01-07 18:54:38 0000000000000002 ffff880833e4d370 ffff8810333c7de0 ffffffff81051439 2014-01-07 18:54:38 ffff88103008e638 ffff8810333c7fd8 000000000000fb88 ffff88103008e638 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:38 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/13 S 000000000000000d 0 5521 2 0x00000000 2014-01-07 18:54:38 ffff881030e3be20 0000000000000046 ffff881030e3bde8 ffff881030e3bde4 2014-01-07 18:54:38 0000000000000003 ffff88083fe82a00 ffff88085c496740 0000000000000207 2014-01-07 18:54:38 ffff881030e39ab8 ffff881030e3bfd8 000000000000fb88 ffff881030e39ab8 2014-01-07 18:54:38 Call Trace: 2014-01-07 18:54:38 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:38 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:38 [] kthread+0x96/0xa0 2014-01-07 18:54:38 [] child_rip+0xa/0x20 2014-01-07 18:54:38 [] ? kthread+0x0/0xa0 2014-01-07 18:54:38 [] ? child_rip+0x0/0x20 2014-01-07 18:54:38 zvol/14 S 000000000000000e 0 5522 2 0x00000000 2014-01-07 18:54:38 ffff881030e3fe20 0000000000000046 ffff881030e3fde8 ffff881030e3fde4 2014-01-07 18:54:38 0000000000000003 ffff88083fe82c00 ffff88085c496740 0000000000000207 2014-01-07 18:54:39 ffff881030e39058 ffff881030e3ffd8 000000000000fb88 ffff881030e39058 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/15 S 000000000000000f 0 5523 2 0x00000000 2014-01-07 18:54:39 ffff8810333f9e20 0000000000000046 ffff8810333f9de8 ffff8810333f9de4 2014-01-07 18:54:39 0000000000000004 ffff88083fe82e00 ffff88085c496740 0000000000000207 2014-01-07 18:54:39 ffff881030e385f8 ffff8810333f9fd8 000000000000fb88 ffff881030e385f8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/16 S 0000000000000000 0 5524 2 0x00000000 2014-01-07 18:54:39 ffff8810333fde20 0000000000000046 0000000000000000 000000000000041e 2014-01-07 18:54:39 0000000000000003 0000000000000001 0000000000000001 0000000000000000 2014-01-07 18:54:39 ffff8810333fbaf8 ffff8810333fdfd8 000000000000fb88 ffff8810333fbaf8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/17 S 0000000000000001 0 5525 2 0x00000000 2014-01-07 18:54:39 ffff881030c6be20 0000000000000046 ffff881030c6bde8 ffff881030c6bde4 2014-01-07 18:54:39 0000000000000002 ffff88083fe81200 ffff88085c496740 00000000000003f8 2014-01-07 18:54:39 ffff8810333fb098 ffff881030c6bfd8 000000000000fb88 ffff8810333fb098 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/18 S 0000000000000002 0 5526 2 0x00000000 2014-01-07 18:54:39 ffff881030c6de20 0000000000000046 ffff881030c6dde8 ffff881030c6dde4 2014-01-07 18:54:39 0000000000000002 ffff88083fe81400 ffff88085c496740 00000000000003f8 2014-01-07 18:54:39 ffff8810333fa638 ffff881030c6dfd8 000000000000fb88 ffff8810333fa638 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/19 S 0000000000000003 0 5527 2 0x00000000 2014-01-07 18:54:39 ffff8810333d9e20 0000000000000046 ffff8810333d9de8 ffff8810333d9de4 2014-01-07 18:54:39 0000000000000002 ffff88083fe81600 ffff88085c496740 00000000000003f8 2014-01-07 18:54:39 ffff881030c6fab8 ffff8810333d9fd8 000000000000fb88 ffff881030c6fab8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/20 S 0000000000000004 0 5528 2 0x00000000 2014-01-07 18:54:39 ffff8810333dde20 0000000000000046 ffff8810333ddde8 ffff8810333ddde4 2014-01-07 18:54:39 0000000000000002 ffff88083fe81800 ffff88085c496740 00000000000003f8 2014-01-07 18:54:39 ffff881030c6f058 ffff8810333ddfd8 000000000000fb88 ffff881030c6f058 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/21 S 0000000000000005 0 5529 2 0x00000000 2014-01-07 18:54:39 ffff8810333dfe20 0000000000000046 0000000000000000 000000000000040f 2014-01-07 18:54:39 0000000000000002 0000000000000001 0000000000000001 0000000000000000 2014-01-07 18:54:39 ffff881030c6e5f8 ffff8810333dffd8 000000000000fb88 ffff881030c6e5f8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/22 S 0000000000000006 0 5530 2 0x00000000 2014-01-07 18:54:39 ffff881030813e20 0000000000000046 0000000000000000 000000000000040f 2014-01-07 18:54:39 0000000000000002 0000000000000001 0000000000000001 0000000000000000 2014-01-07 18:54:39 ffff881030811af8 ffff881030813fd8 000000000000fb88 ffff881030811af8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/23 S 0000000000000007 0 5531 2 0x00000000 2014-01-07 18:54:39 ffff881030817e20 0000000000000046 ffff881030817de8 ffff881030817de4 2014-01-07 18:54:39 0000000000000002 ffff88083fe81e00 ffff88085c496740 0000000000000400 2014-01-07 18:54:39 ffff881030811098 ffff881030817fd8 000000000000fb88 ffff881030811098 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/24 S 0000000000000008 0 5532 2 0x00000000 2014-01-07 18:54:39 ffff881031aebe20 0000000000000046 ffff881031aebde8 ffff881031aebde4 2014-01-07 18:54:39 0000000000000002 ffff88083fe82000 ffff88085c496740 0000000000000207 2014-01-07 18:54:39 ffff881030810638 ffff881031aebfd8 000000000000fb88 ffff881030810638 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/25 S 0000000000000009 0 5533 2 0x00000000 2014-01-07 18:54:39 ffff881031aefe20 0000000000000046 ffff881031aefde8 ffff881031aefde4 2014-01-07 18:54:39 0000000000000002 ffff88083fe82200 ffff88085c496740 0000000000000207 2014-01-07 18:54:39 ffff881031aedab8 ffff881031aeffd8 000000000000fb88 ffff881031aedab8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/26 S 000000000000000a 0 5534 2 0x00000000 2014-01-07 18:54:39 ffff8810300d3e20 0000000000000046 0000000000000082 000000000000040f 2014-01-07 18:54:39 0000000000000002 0000000000000001 0000000000000001 0000000000000000 2014-01-07 18:54:39 ffff881031aed058 ffff8810300d3fd8 000000000000fb88 ffff881031aed058 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/27 S 000000000000000b 0 5535 2 0x00000000 2014-01-07 18:54:39 ffff8810300d5e20 0000000000000046 ffff8810300d5de8 ffff8810300d5de4 2014-01-07 18:54:39 0000000000000002 ffff88083fe82600 ffff88085c496740 0000000000000207 2014-01-07 18:54:39 ffff881031aec5f8 ffff8810300d5fd8 000000000000fb88 ffff881031aec5f8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/28 S 000000000000000c 0 5536 2 0x00000000 2014-01-07 18:54:39 ffff881031931e20 0000000000000046 ffff881031931d90 ffffffff81096db6 2014-01-07 18:54:39 0000000000000002 ffff880833e4d370 ffff881031931de0 ffffffff81051439 2014-01-07 18:54:39 ffff8810300d7af8 ffff881031931fd8 000000000000fb88 ffff8810300d7af8 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:39 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/29 S 000000000000000d 0 5537 2 0x00000000 2014-01-07 18:54:39 ffff881031935e20 0000000000000046 0000000000000000 ffff881031935de4 2014-01-07 18:54:39 0000000000000000 ffff88083fe82a00 ffff88085c496740 000000000000015f 2014-01-07 18:54:39 ffff8810300d7098 ffff881031935fd8 000000000000fb88 ffff8810300d7098 2014-01-07 18:54:39 Call Trace: 2014-01-07 18:54:39 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:39 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:39 [] kthread+0x96/0xa0 2014-01-07 18:54:39 [] child_rip+0xa/0x20 2014-01-07 18:54:39 [] ? kthread+0x0/0xa0 2014-01-07 18:54:39 [] ? child_rip+0x0/0x20 2014-01-07 18:54:39 zvol/30 S 000000000000000e 0 5538 2 0x00000000 2014-01-07 18:54:39 ffff881031937e20 0000000000000046 0000000000000000 ffff881031937de4 2014-01-07 18:54:40 0000000000000000 ffff88083fe82c00 ffff88085c496740 000000000000015f 2014-01-07 18:54:40 ffff8810300d6638 ffff881031937fd8 000000000000fb88 ffff8810300d6638 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 zvol/31 S 000000000000000f 0 5539 2 0x00000000 2014-01-07 18:54:40 ffff8810309c5e20 0000000000000046 0000000000000000 ffff8810309c5de4 2014-01-07 18:54:40 0000000000000000 ffff88083fe82e00 ffff88085c496740 000000000000015f 2014-01-07 18:54:40 ffff8810309c3ab8 ffff8810309c5fd8 000000000000fb88 ffff8810309c3ab8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_connd S 0000000000000001 0 5621 2 0x00000000 2014-01-07 18:54:40 ffff8810332b9db0 0000000000000046 0000000000000000 ffff8810332b9d74 2014-01-07 18:54:40 0000000000000000 0000000000000286 ffff8810332b9d50 ffffffff810810cc 2014-01-07 18:54:40 ffff881030cb6638 ffff8810332b9fd8 000000000000fb88 ffff881030cb6638 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:40 [] kiblnd_connd+0x1de/0x360 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_connd+0x0/0x360 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_00_ S 0000000000000000 0 5622 2 0x00000000 2014-01-07 18:54:40 ffff88103034de00 0000000000000046 0000000000000000 ffff88103034ddc4 2014-01-07 18:54:40 ffffc90000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:40 ffff88103185c638 ffff88103034dfd8 000000000000fb88 ffff88103185c638 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_00_ S 0000000000000000 0 5623 2 0x00000000 2014-01-07 18:54:40 ffff8810333a1e00 0000000000000046 0000000000000000 ffff8810333a1dc4 2014-01-07 18:54:40 0000000000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:40 ffff881030fb1058 ffff8810333a1fd8 000000000000fb88 ffff881030fb1058 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_01_ S 0000000000000007 0 5624 2 0x00000000 2014-01-07 18:54:40 ffff88103416fe00 0000000000000046 0000000000000000 ffff88103416fdc4 2014-01-07 18:54:40 0000000000000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:54:40 ffff8810300e25f8 ffff88103416ffd8 000000000000fb88 ffff8810300e25f8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_01_ S 0000000000000004 0 5625 2 0x00000000 2014-01-07 18:54:40 ffff88103429fe00 0000000000000046 0000000000000000 ffff88103429fdc4 2014-01-07 18:54:40 ffffc90000000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:40 ffff88103038b098 ffff88103429ffd8 000000000000fb88 ffff88103038b098 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_02_ S 0000000000000008 0 5626 2 0x00000000 2014-01-07 18:54:40 ffff881031e05e00 0000000000000046 ffff881031e05da0 ffffffffa068984b 2014-01-07 18:54:40 ffffc90017716000 00000000ffffffff ffff881031e05e00 ffff881004c91200 2014-01-07 18:54:40 ffff881030357058 ffff881031e05fd8 000000000000fb88 ffff881030357058 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_02_ S 000000000000000a 0 5627 2 0x00000000 2014-01-07 18:54:40 ffff8810323cde00 0000000000000046 0000000000000000 ffffffffa068984b 2014-01-07 18:54:40 00000000fffffffa 00000000fffffffd ffff8810323cddc0 ffff880e906ae600 2014-01-07 18:54:40 ffff881031967058 ffff8810323cdfd8 000000000000fb88 ffff881031967058 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] ? kiblnd_handle_rx+0x2cb/0x640 [ko2iblnd] 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_03_ S 000000000000000c 0 5628 2 0x00000000 2014-01-07 18:54:40 ffff881034213e00 0000000000000046 0000000000000000 ffffffffa068984b 2014-01-07 18:54:40 00000000fffffffa 00000000fffffffd ffff881034213dc0 ffff880e906ae400 2014-01-07 18:54:40 ffff881031967ab8 ffff881034213fd8 000000000000fb88 ffff881031967ab8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 kiblnd_sd_03_ S 000000000000000d 0 5629 2 0x00000000 2014-01-07 18:54:40 ffff881032cd9e00 0000000000000046 0000000000000000 ffffffffa068984b 2014-01-07 18:54:40 ffffc90017716000 00000000ffffffff ffff881032cd9e00 ffff880e906ae400 2014-01-07 18:54:40 ffff88103185daf8 ffff881032cd9fd8 000000000000fb88 ffff88103185daf8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] ? kiblnd_handle_rx+0x2cb/0x640 [ko2iblnd] 2014-01-07 18:54:40 [] kiblnd_scheduler+0x3b6/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:40 [] ? kiblnd_scheduler+0x0/0x7b0 [ko2iblnd] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 router_checke S 0000000000000001 0 5630 2 0x00000000 2014-01-07 18:54:40 ffff8810071e3d80 0000000000000046 0000000000000000 00000000a03c84e0 2014-01-07 18:54:40 0000000000000000 ffff8807ba1784af 0000005100000004 ffffffffa0636c58 2014-01-07 18:54:40 ffff881030fb05f8 ffff8810071e3fd8 000000000000fb88 ffff881030fb05f8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:40 [] lnet_router_checker+0x47a/0x590 [lnet] 2014-01-07 18:54:40 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:40 [] ? lnet_router_checker+0x0/0x590 [lnet] 2014-01-07 18:54:40 [] kthread+0x96/0xa0 2014-01-07 18:54:40 [] child_rip+0xa/0x20 2014-01-07 18:54:40 [] ? kthread+0x0/0xa0 2014-01-07 18:54:40 [] ? child_rip+0x0/0x20 2014-01-07 18:54:40 crond S 0000000000000008 0 5637 1 0x00000000 2014-01-07 18:54:40 ffff88080e1a9e68 0000000000000086 0000000000000000 ffffffff8109bc05 2014-01-07 18:54:40 ffff880800000008 0000000181192a42 ffff8808336234c0 ffff880830770c80 2014-01-07 18:54:40 ffff880834341af8 ffff88080e1a9fd8 000000000000fb88 ffff880834341af8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] ? __hrtimer_start_range_ns+0x1a5/0x470 2014-01-07 18:54:40 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:40 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:40 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:40 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:40 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:40 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:40 cerebrod S 0000000000000002 0 5646 1 0x00000000 2014-01-07 18:54:40 ffff881030d0fe68 0000000000000086 0000000000000000 ffff881030d0fe2c 2014-01-07 18:54:40 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:40 ffff881030378638 ffff881030d0ffd8 000000000000fb88 ffff881030378638 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] do_nanosleep+0x8b/0xc0 2014-01-07 18:54:40 [] hrtimer_nanosleep+0xc4/0x180 2014-01-07 18:54:40 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:40 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:40 [] sys_nanosleep+0x74/0x80 2014-01-07 18:54:40 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:40 stapio S 0000000000000001 0 6435 1 0x00000000 2014-01-07 18:54:40 ffff8810321838d8 0000000000000082 0000000000000000 ffffea001c4993d8 2014-01-07 18:54:40 0000000000000000 000200da00000000 ffff880000021ea8 0000000000000030 2014-01-07 18:54:40 ffff881030063af8 ffff881032183fd8 000000000000fb88 ffff881030063af8 2014-01-07 18:54:40 Call Trace: 2014-01-07 18:54:40 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:54:40 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:54:40 [] ? __pollwait+0x75/0xf0 2014-01-07 18:54:40 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:40 [] do_select+0x57c/0x6c0 2014-01-07 18:54:40 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:40 [] ? pollwake+0x0/0x60 2014-01-07 18:54:40 [] ? __stp_relay_dropped_open+0x0/0x10 [toss2361] 2014-01-07 18:54:40 [] ? vga_set_mem_top+0x59/0x60 2014-01-07 18:54:40 [] ? vgacon_scroll+0x107/0x2e0 2014-01-07 18:54:41 [] ? vgacon_set_cursor_size+0xbd/0x150 2014-01-07 18:54:41 [] ? notifier_call_chain+0x16/0x80 2014-01-07 18:54:41 [] ? perf_event_task_sched_out+0x33/0x80 2014-01-07 18:54:41 [] ? notify_update+0x2e/0x30 2014-01-07 18:54:41 [] ? vt_console_print+0x260/0x330 2014-01-07 18:54:41 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:54:41 [] ? __switch_to+0x1ac/0x320 2014-01-07 18:54:41 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:54:41 [] ? vprintk+0x251/0x560 2014-01-07 18:54:41 [] ? cache_alloc_refill+0x1c0/0x240 2014-01-07 18:54:41 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:41 [] ? _stp_ctl_read_cmd+0x8f/0x260 [toss2361] 2014-01-07 18:54:41 [] sys_pselect6+0x5e/0x210 2014-01-07 18:54:41 [] ? security_file_fcntl+0x16/0x20 2014-01-07 18:54:41 [] ? sys_fcntl+0x118/0x530 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 stapio S 0000000000000002 0 6436 1 0x00000000 2014-01-07 18:54:41 ffff88080e367dc8 0000000000000082 ffff88080e367d90 ffff88080e367d8c 2014-01-07 18:54:41 0000000000000000 ffff88083fe81400 ffff880044636740 0000000000000400 2014-01-07 18:54:41 ffff880831907ab8 ffff88080e367fd8 000000000000fb88 ffff880831907ab8 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:54:41 [] schedule_timeout_interruptible+0x1e/0x20 2014-01-07 18:54:41 [] sys_rt_sigtimedwait+0x206/0x260 2014-01-07 18:54:41 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 stapio S 0000000000000008 0 6437 1 0x00000000 2014-01-07 18:54:41 ffff880828379a08 0000000000000082 ffff8808283799d0 ffff8808283799cc 2014-01-07 18:54:41 0000000000000008 ffff88083fe82000 ffff880044636740 0000000000000207 2014-01-07 18:54:41 ffff8808335e65f8 ffff880828379fd8 000000000000fb88 ffff8808335e65f8 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:41 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:54:41 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:54:41 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:54:41 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:54:41 [] do_sys_poll+0x457/0x520 2014-01-07 18:54:41 [] ? __pollwait+0x0/0xf0 2014-01-07 18:54:41 [] ? pollwake+0x0/0x60 2014-01-07 18:54:41 [] ? filemap_fault+0xd3/0x500 2014-01-07 18:54:41 [] ? unlock_page+0x27/0x30 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] ? handle_pte_fault+0xf7/0xb50 2014-01-07 18:54:41 [] ? __lru_cache_add+0x40/0x90 2014-01-07 18:54:41 [] ? lru_cache_add_lru+0x21/0x40 2014-01-07 18:54:41 [] ? page_add_new_anon_rmap+0x9d/0xf0 2014-01-07 18:54:41 [] ? do_huge_pmd_anonymous_page+0x255/0x380 2014-01-07 18:54:41 [] ? handle_mm_fault+0x23a/0x310 2014-01-07 18:54:41 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:54:41 [] ? call_function_single_interrupt+0xe/0x20 2014-01-07 18:54:41 [] ? relay_file_read+0x0/0x2c0 2014-01-07 18:54:41 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:41 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:41 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:41 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:41 [] ? poll_select_set_timeout+0x8d/0xa0 2014-01-07 18:54:41 [] sys_ppoll+0x4c/0x180 2014-01-07 18:54:41 [] ? fput+0xe/0x30 2014-01-07 18:54:41 [] ? sys_read+0x7c/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 agetty S 0000000000000000 0 6445 1 0x00000000 2014-01-07 18:54:41 ffff880831047cc8 0000000000000086 0000000000000000 ffffffff811436a9 2014-01-07 18:54:41 000000082740c025 0000000000000000 0000000000000000 ffffea0038aeeb10 2014-01-07 18:54:41 ffff880832811098 ffff880831047fd8 000000000000fb88 ffff880832811098 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? uart_write+0xe9/0x120 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] ? do_device_not_available+0xe/0x10 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000007 0 6446 1 0x00000000 2014-01-07 18:54:41 ffff88080e1b1cc8 0000000000000082 ffff88080e1b1c28 ffff881033237578 2014-01-07 18:54:41 ffff8800446767a8 ffff880044676740 0000000000000007 0000000000000003 2014-01-07 18:54:41 ffff880828e4fab8 ffff88080e1b1fd8 000000000000fb88 ffff880828e4fab8 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000000 0 6448 1 0x00000000 2014-01-07 18:54:41 ffff88103312bcc8 0000000000000082 ffff88103312bc28 ffff881030914ad8 2014-01-07 18:54:41 ffff8800446567a8 ffff880044656740 0000000000000000 0000000000000002 2014-01-07 18:54:41 ffff881030239af8 ffff88103312bfd8 000000000000fb88 ffff881030239af8 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000003 0 6450 1 0x00000000 2014-01-07 18:54:41 ffff881030b35cc8 0000000000000082 ffff881030b35ca8 ffffffff811436a9 2014-01-07 18:54:41 00000008274aa025 0000000000000000 0000000000000000 ffffea001cb6b5d8 2014-01-07 18:54:41 ffff881033237af8 ffff881030b35fd8 000000000000fb88 ffff881033237af8 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000002 0 6452 1 0x00000000 2014-01-07 18:54:41 ffff8810301dbcc8 0000000000000086 ffff8810301dbca8 ffffffff811436a9 2014-01-07 18:54:41 00000008274aa025 0000000000000000 0000000000000000 ffffea001c31d440 2014-01-07 18:54:41 ffff881030915058 ffff8810301dbfd8 000000000000fb88 ffff881030915058 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000001 0 6454 1 0x00000000 2014-01-07 18:54:41 ffff8810332f5cc8 0000000000000086 ffff8810332f5ca8 ffffffff811436a9 2014-01-07 18:54:41 00000008274aa025 0000000000000000 0000000000000000 ffffea001c2f61f8 2014-01-07 18:54:41 ffff881030bf8638 ffff8810332f5fd8 000000000000fb88 ffff881030bf8638 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 mingetty S 0000000000000001 0 6456 1 0x00000000 2014-01-07 18:54:41 ffff8810319ebcc8 0000000000000086 0000000000000000 ffffffff811436a9 2014-01-07 18:54:41 00000008274aa025 0000000000000000 0000000000000000 ffffea001c319f50 2014-01-07 18:54:41 ffff881030d8d098 ffff8810319ebfd8 000000000000fb88 ffff881030d8d098 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? __do_fault+0x469/0x530 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] ? flush_work+0x21/0xc0 2014-01-07 18:54:41 [] ? try_to_del_timer_sync+0x7b/0xe0 2014-01-07 18:54:41 [] n_tty_read+0x5f7/0x9b0 2014-01-07 18:54:41 [] ? release_console_sem+0x1cf/0x220 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] tty_read+0xa6/0xf0 2014-01-07 18:54:41 [] vfs_read+0xb5/0x1a0 2014-01-07 18:54:41 [] sys_read+0x51/0x90 2014-01-07 18:54:41 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:54:41 lockd S 0000000000000001 0 6593 2 0x00000000 2014-01-07 18:54:41 ffff88102f4a7d10 0000000000000046 ffff88102f4a7dd8 ffff88102f4a7e18 2014-01-07 18:54:41 0000000000000000 0000000000000000 0000000000000000 ffffffffa019e530 2014-01-07 18:54:41 ffff881030379098 ffff88102f4a7fd8 000000000000fb88 ffff881030379098 2014-01-07 18:54:41 Call Trace: 2014-01-07 18:54:41 [] ? rpc_async_release+0x0/0x20 [sunrpc] 2014-01-07 18:54:41 [] schedule_timeout+0x215/0x2e0 2014-01-07 18:54:41 [] svc_recv+0x5a5/0x850 [sunrpc] 2014-01-07 18:54:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:41 [] ? queue_delayed_work+0x21/0x40 2014-01-07 18:54:41 [] lockd+0xc1/0x230 [lockd] 2014-01-07 18:54:41 [] ? lockd+0x0/0x230 [lockd] 2014-01-07 18:54:41 [] kthread+0x96/0xa0 2014-01-07 18:54:41 [] child_rip+0xa/0x20 2014-01-07 18:54:41 [] ? kthread+0x0/0xa0 2014-01-07 18:54:41 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_null_iss/0 S 0000000000000000 0 6744 2 0x00000000 2014-01-07 18:54:42 ffff8810341e7e20 0000000000000046 0000000000000000 ffff8810341e7de4 2014-01-07 18:54:42 ffff881000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:42 ffff881030239098 ffff8810341e7fd8 000000000000fb88 ffff881030239098 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_null_int/0 S 0000000000000000 0 6745 2 0x00000000 2014-01-07 18:54:42 ffff881031d33e20 0000000000000046 0000000000000000 ffff880681ef67f8 2014-01-07 18:54:42 ffff881031d33db0 ffff88070aaed1a8 0000000000100000 ffff88070aaed218 2014-01-07 18:54:42 ffff88103006a5f8 ffff881031d33fd8 000000000000fb88 ffff88103006a5f8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/0 S 0000000000000000 0 6746 2 0x00000000 2014-01-07 18:54:42 ffff88102f589e20 0000000000000046 ffff880e86e4cf00 ffff880e86e4d218 2014-01-07 18:54:42 ffff88102f589db0 ffffffffa0554dc3 0000000000000000 ffff880e86e4cf00 2014-01-07 18:54:42 ffff881030238638 ffff88102f589fd8 000000000000fb88 ffff881030238638 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/1 S 0000000000000001 0 6747 2 0x00000000 2014-01-07 18:54:42 ffff881030153e20 0000000000000046 0000000000000000 ffff88070a7707b8 2014-01-07 18:54:42 ffff881030153db0 ffffffffa0554dc3 0000000000000000 ffff88070a7704a0 2014-01-07 18:54:42 ffff881030bf9098 ffff881030153fd8 000000000000fb88 ffff881030bf9098 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/2 S 0000000000000002 0 6748 2 0x00000000 2014-01-07 18:54:42 ffff881030ff3e20 0000000000000046 0000000000000000 ffff88070a775218 2014-01-07 18:54:42 ffff881030ff3db0 ffffffffa0554dc3 0000000000000000 ffff88070a774f00 2014-01-07 18:54:42 ffff881030063098 ffff881030ff3fd8 000000000000fb88 ffff881030063098 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/3 S 0000000000000003 0 6749 2 0x00000000 2014-01-07 18:54:42 ffff88102f68be20 0000000000000046 0000000000000000 ffff88078c9f7dd8 2014-01-07 18:54:42 ffff88102f68bdb0 ffffffffa0554dc3 0000000000000000 ffff88078c9f7ac0 2014-01-07 18:54:42 ffff881030961ab8 ffff88102f68bfd8 000000000000fb88 ffff881030961ab8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/4 S 0000000000000004 0 6750 2 0x00000000 2014-01-07 18:54:42 ffff88103090fe20 0000000000000046 ffff880e9248dc20 ffff880e9248df38 2014-01-07 18:54:42 ffff88103090fdb0 ffffffffa0554dc3 0000000000000000 ffff880e9248dc20 2014-01-07 18:54:42 ffff881031b8e638 ffff88103090ffd8 000000000000fb88 ffff881031b8e638 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/5 S 0000000000000005 0 6751 2 0x00000000 2014-01-07 18:54:42 ffff8810331e9e20 0000000000000046 0000000000000000 ffff88078efbaf18 2014-01-07 18:54:42 ffff8810331e9db0 ffffffffa0554dc3 0000000000000000 ffff88078efbac00 2014-01-07 18:54:42 ffff881037d46638 ffff8810331e9fd8 000000000000fb88 ffff881037d46638 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/6 S 0000000000000006 0 6752 2 0x00000000 2014-01-07 18:54:42 ffff88103190fe20 0000000000000046 0000000000000000 ffff880e8c6d87b8 2014-01-07 18:54:42 ffff88103190fdb0 ffffffffa0554dc3 0000000000000000 ffff880e8c6d84a0 2014-01-07 18:54:42 ffff881031a5daf8 ffff88103190ffd8 000000000000fb88 ffff881031a5daf8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_iss/7 S 0000000000000007 0 6753 2 0x00000000 2014-01-07 18:54:42 ffff8810347f7e20 0000000000000046 ffff8810347f7de8 ffff8810347f7de4 2014-01-07 18:54:42 ffff8810347f7db0 ffff88083fe81e00 ffff88085c4d6740 000000000000af70 2014-01-07 18:54:42 ffff881030d8daf8 ffff8810347f7fd8 000000000000fb88 ffff881030d8daf8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/0 S 0000000000000000 0 6754 2 0x00000000 2014-01-07 18:54:42 ffff881031b47e20 0000000000000046 0000000000000000 ffff881031b47de4 2014-01-07 18:54:42 00000a5a00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:42 ffff881030d8c638 ffff881031b47fd8 000000000000fb88 ffff881030d8c638 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/1 R running task 0 6755 2 0x00000000 2014-01-07 18:54:42 ffff881030b91e20 0000000000000046 0000000000000000 ffff880e7ce437d8 2014-01-07 18:54:42 00000a5d37102000 ffff880e7ce42188 0000000000100000 ffff880e7ce421f8 2014-01-07 18:54:42 ffff88103031a638 ffff881030b91fd8 000000000000fb88 ffff88103031a638 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/2 S 0000000000000002 0 6756 2 0x00000000 2014-01-07 18:54:42 ffff881030b37e20 0000000000000046 0000000000000000 ffff881030b37de4 2014-01-07 18:54:42 00000a5c00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:42 ffff88102f4d7ab8 ffff881030b37fd8 000000000000fb88 ffff88102f4d7ab8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/3 S 0000000000000003 0 6757 2 0x00000000 2014-01-07 18:54:42 ffff8810319e7e20 0000000000000046 0000000000000000 ffff8810319e7de4 2014-01-07 18:54:42 00000a5b00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:42 ffff88102f4d65f8 ffff8810319e7fd8 000000000000fb88 ffff8807b509caa0 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/4 S 0000000000000004 0 6758 2 0x00000000 2014-01-07 18:54:42 ffff8810347c3e20 0000000000000046 0000000000000000 ffff8810347c3de4 2014-01-07 18:54:42 00000a5c00000000 ffff88083fe81800 ffff88085c436740 000000000000015a 2014-01-07 18:54:42 ffff88103031baf8 ffff8810347c3fd8 000000000000fb88 ffff88103031baf8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/5 S 0000000000000005 0 6759 2 0x00000000 2014-01-07 18:54:42 ffff8810309e7e20 0000000000000046 0000000000000000 ffff8810309e7de4 2014-01-07 18:54:42 00000a5a00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:54:42 ffff8810301ddab8 0000000000000000 000000000000fb88 ffff8810301ddab8 2014-01-07 18:54:42 Call Trace: 2014-01-07 18:54:42 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:42 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:42 [] kthread+0x96/0xa0 2014-01-07 18:54:42 [] child_rip+0xa/0x20 2014-01-07 18:54:42 [] ? kthread+0x0/0xa0 2014-01-07 18:54:42 [] ? child_rip+0x0/0x20 2014-01-07 18:54:42 z_rd_int/6 S 0000000000000006 0 6760 2 0x00000000 2014-01-07 18:54:43 ffff88103030fe20 0000000000000046 0000000000000000 ffff88103030fde4 2014-01-07 18:54:43 00000a5b00000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:43 ffff8810301dd058 ffff88103030ffd8 000000000000fb88 ffff8810301dd058 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/7 S 0000000000000007 0 6761 2 0x00000000 2014-01-07 18:54:43 ffff881030f2fe20 0000000000000046 0000000000000000 ffff881030f2fde4 2014-01-07 18:54:43 00000a5900000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:54:43 ffff8810301dc5f8 ffff881030f2ffd8 000000000000fb88 ffff8810301dc5f8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/8 S 0000000000000008 0 6762 2 0x00000000 2014-01-07 18:54:43 ffff881030c85e20 0000000000000046 ffff881030c85de8 ffff881030c85de4 2014-01-07 18:54:43 00000a5d77606000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff881034dc1af8 ffff881030c85fd8 000000000000fb88 ffff881034dc1af8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/9 S 0000000000000009 0 6763 2 0x00000000 2014-01-07 18:54:43 ffff881031a07e20 0000000000000046 0000000000000000 ffff881031a07de4 2014-01-07 18:54:43 00000a5c00000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff881034dc1098 ffff881031a07fd8 000000000000fb88 ffff881034dc1098 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/10 S 000000000000000a 0 6764 2 0x00000000 2014-01-07 18:54:43 ffff881030d7fe20 0000000000000046 0000000000000000 ffff881030d7fde4 2014-01-07 18:54:43 00000a5c00000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff881034dc0638 ffff881030d7ffd8 000000000000fb88 ffff881034dc0638 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/11 S 000000000000000b 0 6765 2 0x00000000 2014-01-07 18:54:43 ffff881030a13e20 0000000000000046 0000000000000000 ffff881030a13de4 2014-01-07 18:54:43 00000a5b00000000 ffff88083fe82600 ffff880044636740 ffff880e7ce294d8 2014-01-07 18:54:43 ffff881030163ab8 ffff881030a13fd8 000000000000fb88 ffff881030163ab8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/12 S 000000000000000c 0 6766 2 0x00000000 2014-01-07 18:54:43 ffff881031bf5e20 0000000000000046 0000000000000000 ffff881031bf5de4 2014-01-07 18:54:43 00000a5900000000 ffff880e8c9e7048 0000000000100000 ffff880e8c9e70b8 2014-01-07 18:54:43 ffff881030163058 ffff881031bf5fd8 000000000000fb88 ffff881030163058 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/13 S 000000000000000d 0 6767 2 0x00000000 2014-01-07 18:54:43 ffff881030c83e20 0000000000000046 0000000000000000 ffff881030c83de4 2014-01-07 18:54:43 00000a5900000000 ffff88083fe82a00 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff8810301625f8 ffff881030c83fd8 000000000000fb88 ffff8810301625f8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/14 S 000000000000000e 0 6768 2 0x00000000 2014-01-07 18:54:43 ffff881030d7be20 0000000000000046 0000000000000000 ffff881030d7bde4 2014-01-07 18:54:43 00000a5a00000000 ffff88083fe82c00 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff8810323ffaf8 ffff881030d7bfd8 000000000000fb88 ffff8810323ffaf8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_rd_int/15 S 000000000000000f 0 6769 2 0x00000000 2014-01-07 18:54:43 ffff8810320a1e20 0000000000000046 ffff8810320a1de8 ffff8810320a1de4 2014-01-07 18:54:43 00000a5b00000000 ffff88083fe82e00 ffff880044636740 00000000000001ff 2014-01-07 18:54:43 ffff8810323ff098 ffff8810320a1fd8 000000000000fb88 ffff8810323ff098 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/0 S 0000000000000000 0 6770 2 0x00000000 2014-01-07 18:54:43 ffff88102f4abe20 0000000000000046 ffff880e8cc85360 ffff880e8cc85678 2014-01-07 18:54:43 ffff88102f4abdb0 ffffffffa0554dc3 0000000000000000 ffff880e8cc85360 2014-01-07 18:54:43 ffff8810323fe638 ffff88102f4abfd8 000000000000fb88 ffff8810323fe638 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/1 S 0000000000000001 0 6771 2 0x00000000 2014-01-07 18:54:43 ffff88103022be20 0000000000000046 0000000000000000 ffff88103022bde4 2014-01-07 18:54:43 0000000000000000 ffff88083fe81200 ffff880044616740 000000000000000f 2014-01-07 18:54:43 ffff881031ae1ab8 ffff88103022bfd8 000000000000fb88 ffff881031ae1ab8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/2 S 0000000000000002 0 6772 2 0x00000000 2014-01-07 18:54:43 ffff881030837e20 0000000000000046 0000000000000000 ffff881030837de4 2014-01-07 18:54:43 0000000000000000 ffff88083fe81400 ffff88085c436740 000000000000ab7d 2014-01-07 18:54:43 ffff881031ae1058 ffff881030837fd8 000000000000fb88 ffff881031ae1058 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/3 S 0000000000000003 0 6773 2 0x00000000 2014-01-07 18:54:43 ffff88103198de20 0000000000000046 0000000000000000 ffff88103198dde4 2014-01-07 18:54:43 0000000000000000 ffff88083fe81600 ffff88085c436740 000000000000ab65 2014-01-07 18:54:43 ffff881031ae05f8 ffff88103198dfd8 000000000000fb88 ffff881031ae05f8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/4 S 0000000000000004 0 6774 2 0x00000000 2014-01-07 18:54:43 ffff881030a85e20 0000000000000046 ffff880e8c1b4640 ffff880e8c1b4958 2014-01-07 18:54:43 ffff881030a85db0 ffffffffa0554dc3 0000000000000000 ffff880e8c1b4640 2014-01-07 18:54:43 ffff881030b9baf8 ffff881030a85fd8 000000000000fb88 ffff881030b9baf8 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] ? zio_wait_for_children+0x63/0x80 [zfs] 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:43 [] ? kthread+0x0/0xa0 2014-01-07 18:54:43 [] ? child_rip+0x0/0x20 2014-01-07 18:54:43 z_wr_iss/5 S 0000000000000005 0 6775 2 0x00000000 2014-01-07 18:54:43 ffff8810341e9e20 0000000000000046 0000000000000000 ffff8810341e9de4 2014-01-07 18:54:43 ffff881000000000 ffff88083fe81a00 ffff880044636740 0000000000000207 2014-01-07 18:54:43 ffff881030b9b098 ffff8810341e9fd8 000000000000fb88 ffff881030b9b098 2014-01-07 18:54:43 Call Trace: 2014-01-07 18:54:43 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:43 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:43 [] kthread+0x96/0xa0 2014-01-07 18:54:43 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/6 S 0000000000000006 0 6776 2 0x00000000 2014-01-07 18:54:44 ffff8810309b5e20 0000000000000046 ffff8810309b5db0 ffffffffa0503e9b 2014-01-07 18:54:44 0000000000004000 ffff880e8e90b920 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:44 ffff881030b9a638 ffff8810309b5fd8 000000000000fb88 ffff881030b9a638 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/7 S 0000000000000007 0 6777 2 0x00000000 2014-01-07 18:54:44 ffff88103019fe20 0000000000000046 0000000000000000 ffff88103019fde4 2014-01-07 18:54:44 0000000000000000 ffff88083fe81e00 ffff880044616740 000000000000000f 2014-01-07 18:54:44 ffff881031951ab8 ffff88103019ffd8 000000000000fb88 ffff881031951ab8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/8 S 0000000000000008 0 6778 2 0x00000000 2014-01-07 18:54:44 ffff8810342e5e20 0000000000000046 0000000000000000 ffff8810342e5de4 2014-01-07 18:54:44 0000000000000000 ffff88083fe82000 ffff880044616740 0000000000000207 2014-01-07 18:54:44 ffff881031951058 ffff8810342e5fd8 000000000000fb88 ffff881031951058 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/9 S 0000000000000009 0 6779 2 0x00000000 2014-01-07 18:54:44 ffff881031899e20 0000000000000046 0000000000000000 ffff881031899de4 2014-01-07 18:54:44 ffff881000000000 ffff88083fe82200 ffff880044616740 00000000000001ff 2014-01-07 18:54:44 ffff8810319505f8 ffff881031899fd8 000000000000fb88 ffff8810319505f8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/10 S 000000000000000a 0 6780 2 0x00000000 2014-01-07 18:54:44 ffff88103191be20 0000000000000046 0000000000000000 ffff88103191bde4 2014-01-07 18:54:44 ffff881000000000 ffff88083fe82400 ffff880044616740 0000000000000207 2014-01-07 18:54:44 ffff881037d4daf8 ffff88103191bfd8 000000000000fb88 ffff881037d4daf8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/11 S 000000000000000b 0 6781 2 0x00000000 2014-01-07 18:54:44 ffff881031a1be20 0000000000000046 ffff881031a1bde8 ffff881031a1bde4 2014-01-07 18:54:44 ffff881031a1bdb0 ffff88083fe82600 ffff88085c436740 000000000000ad64 2014-01-07 18:54:44 ffff881037d4d098 ffff881031a1bfd8 000000000000fb88 ffff881037d4d098 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/12 S 000000000000000c 0 6782 2 0x00000000 2014-01-07 18:54:44 ffff881030b0be20 0000000000000046 ffff881030b0bde8 ffff881030b0bde4 2014-01-07 18:54:44 ffff881030b0bdb0 ffff88083fe82800 ffff88085c436740 000000000000ad64 2014-01-07 18:54:44 ffff881037d4c638 ffff881030b0bfd8 000000000000fb88 ffff881037d4c638 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/13 S 000000000000000d 0 6783 2 0x00000000 2014-01-07 18:54:44 ffff8810309bbe20 0000000000000046 0000000000000000 ffff8810309bbde4 2014-01-07 18:54:44 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000207 2014-01-07 18:54:44 ffff881030a5dab8 ffff8810309bbfd8 000000000000fb88 ffff881030a5dab8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/14 S 000000000000000e 0 6784 2 0x00000000 2014-01-07 18:54:44 ffff881030ee1e20 0000000000000046 0000000000000000 ffff881030ee1de4 2014-01-07 18:54:44 ffff881000000000 ffff88083fe82c00 ffff880044616740 0000000000000207 2014-01-07 18:54:44 ffff881030a5d058 ffff881030ee1fd8 000000000000fb88 ffff881030a5d058 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss/15 S 000000000000000f 0 6785 2 0x00000000 2014-01-07 18:54:44 ffff881030e0fe20 0000000000000046 0000000000000000 ffff881030e0fde4 2014-01-07 18:54:44 0000000000000000 ffff88083fe82e00 ffff880044616740 0000000000000207 2014-01-07 18:54:44 ffff881030a5c5f8 ffff881030e0ffd8 000000000000fb88 ffff881030a5c5f8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss_h/0 S 0000000000000000 0 6786 2 0x00000000 2014-01-07 18:54:44 ffff8810318efe20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:44 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:44 ffff88103002daf8 ffff8810318effd8 000000000000fb88 ffff88103002daf8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss_h/1 S 0000000000000001 0 6787 2 0x00000000 2014-01-07 18:54:44 ffff881034dc5e20 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:44 0000000000000000 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:54:44 ffff88103002d098 ffff881034dc5fd8 000000000000fb88 ffff88103002d098 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss_h/2 S 0000000000000002 0 6788 2 0x00000000 2014-01-07 18:54:44 ffff881030bc9e20 0000000000000046 0000000000000000 ffff88103002c0b8 2014-01-07 18:54:44 0000000000000000 ffff8800446367a8 0000000000000001 ffff880044636740 2014-01-07 18:54:44 ffff88103002c638 ffff881030bc9fd8 000000000000fb88 ffff88103002c638 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss_h/3 S 0000000000000003 0 6789 2 0x00000000 2014-01-07 18:54:44 ffff881030b2de20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:44 0000000000000000 ffff8808270bfa30 ffff881030b2dde0 ffffffff81051439 2014-01-07 18:54:44 ffff881030bcbab8 ffff881030b2dfd8 000000000000fb88 ffff881030bcbab8 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:44 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_iss_h/4 S 0000000000000004 0 6790 2 0x00000000 2014-01-07 18:54:44 ffff881030a4de20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:44 0000000000000000 ffff8808270bfa30 ffff881030a4dde0 ffffffff81051439 2014-01-07 18:54:44 ffff881030bcb058 ffff881030a4dfd8 000000000000fb88 ffff881030bcb058 2014-01-07 18:54:44 Call Trace: 2014-01-07 18:54:44 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:44 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:44 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:44 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:44 [] kthread+0x96/0xa0 2014-01-07 18:54:44 [] child_rip+0xa/0x20 2014-01-07 18:54:44 [] ? kthread+0x0/0xa0 2014-01-07 18:54:44 [] ? child_rip+0x0/0x20 2014-01-07 18:54:44 z_wr_int/0 S 0000000000000000 0 6791 2 0x00000000 2014-01-07 18:54:44 ffff881030a4fe20 0000000000000046 0000000000000000 ffff8806ded087b8 2014-01-07 18:54:44 ffff881030a4fdb0 ffff880e909fcd48 0000000000100000 ffff880e909fcdb8 2014-01-07 18:54:44 ffff881030bca5f8 ffff881030a4ffd8 000000000000fb88 ffff881030bca5f8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/1 S 0000000000000001 0 6792 2 0x00000000 2014-01-07 18:54:45 ffff8810331b3e20 0000000000000046 0000000000000000 ffff880e8f9c2658 2014-01-07 18:54:45 ffff8810331b3db0 ffff880e80943768 0000000000100000 ffff880e809437d8 2014-01-07 18:54:45 ffff8810331b1af8 ffff8810331b3fd8 000000000000fb88 ffff8810331b1af8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/2 S 0000000000000002 0 6793 2 0x00000000 2014-01-07 18:54:45 ffff881037d4be20 0000000000000046 0000000000000000 ffff880e86d014d8 2014-01-07 18:54:45 ffff881037d4bdb0 ffff88050f320ba8 0000000000100000 ffff88050f320c18 2014-01-07 18:54:45 ffff8810331b1098 ffff881037d4bfd8 000000000000fb88 ffff8810331b1098 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/3 S 0000000000000003 0 6794 2 0x00000000 2014-01-07 18:54:45 ffff8810331bde20 0000000000000046 0000000000000000 ffff880e8d0d3378 2014-01-07 18:54:45 ffff8810331bddb0 ffff8806deff3308 0000000000100000 ffff8806deff3378 2014-01-07 18:54:45 ffff8810331b0638 ffff8810331bdfd8 000000000000fb88 ffff8810331b0638 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/4 S 0000000000000004 0 6795 2 0x00000000 2014-01-07 18:54:45 ffff881031a27e20 0000000000000046 0000000000000000 ffff880e80808c18 2014-01-07 18:54:45 ffff881031a27db0 ffff880e8c515ec8 0000000000100000 ffff880e8c515f38 2014-01-07 18:54:45 ffff881031a25ab8 ffff881031a27fd8 000000000000fb88 ffff881031a25ab8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/5 S 0000000000000005 0 6796 2 0x00000000 2014-01-07 18:54:45 ffff881030b07e20 0000000000000046 ffff880e7d460040 ffff880e7d460358 2014-01-07 18:54:45 ffff881030b07db0 ffff8804be623768 0000000000100000 ffff8804be6237d8 2014-01-07 18:54:45 ffff881031a25058 ffff881030b07fd8 000000000000fb88 ffff881031a25058 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/6 S 0000000000000006 0 6797 2 0x00000000 2014-01-07 18:54:45 ffff881031b41e20 0000000000000046 0000000000000000 ffff880e7cca87b8 2014-01-07 18:54:45 ffff881031b41db0 ffff880e7c8798c8 0000000000100000 ffff880e7c879938 2014-01-07 18:54:45 ffff881031a245f8 ffff881031b41fd8 000000000000fb88 ffff881031a245f8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/7 S 0000000000000007 0 6798 2 0x00000000 2014-01-07 18:54:45 ffff881030e09e20 0000000000000046 0000000000000000 ffff880615d02ab8 2014-01-07 18:54:45 ffff881030e09db0 ffff8804be623768 0000000000100000 ffff8804be6237d8 2014-01-07 18:54:45 ffff881031b43af8 ffff881030e09fd8 000000000000fb88 ffff881031b43af8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/8 S 0000000000000008 0 6799 2 0x00000000 2014-01-07 18:54:45 ffff881030ee5e20 0000000000000046 0000000000000000 ffff880e8f6e3378 2014-01-07 18:54:45 ffff881030ee5db0 ffff880e7cb3c8e8 0000000000100000 ffff880e7cb3c958 2014-01-07 18:54:45 ffff881031b43098 ffff881030ee5fd8 000000000000fb88 ffff881031b43098 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/9 S 0000000000000009 0 6800 2 0x00000000 2014-01-07 18:54:45 ffff881030ee7e20 0000000000000046 ffff880e7cb31620 ffff880e7cb31938 2014-01-07 18:54:45 ffff881030ee7db0 ffff880e90331d28 0000000000100000 ffff880e90331d98 2014-01-07 18:54:45 ffff881031b42638 ffff881030ee7fd8 000000000000fb88 ffff881031b42638 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/10 S 000000000000000a 0 6801 2 0x00000000 2014-01-07 18:54:45 ffff8810319e3e20 0000000000000046 0000000000000000 ffff880e8d6ffdd8 2014-01-07 18:54:45 ffff8810319e3db0 ffff880e8cd648e8 0000000000100000 ffff880e8cd64958 2014-01-07 18:54:45 ffff8810319e1ab8 ffff8810319e3fd8 000000000000fb88 ffff8810319e1ab8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/11 S 000000000000000b 0 6802 2 0x00000000 2014-01-07 18:54:45 ffff881030029e20 0000000000000046 ffff881030029de8 ffff881030029de4 2014-01-07 18:54:45 ffff881030029db0 ffff88083fe82600 ffff88085c436740 000000000000af5c 2014-01-07 18:54:45 ffff8810319e1058 ffff881030029fd8 000000000000fb88 ffff8810319e1058 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/12 S 000000000000000c 0 6803 2 0x00000000 2014-01-07 18:54:45 ffff88103002be20 0000000000000046 ffff880787da7660 ffff880787da7978 2014-01-07 18:54:45 ffff88103002bdb0 ffff8806dec31008 0000000000100000 ffff8806dec31078 2014-01-07 18:54:45 ffff8810319e05f8 ffff88103002bfd8 000000000000fb88 ffff8810319e05f8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/13 S 000000000000000d 0 6804 2 0x00000000 2014-01-07 18:54:45 ffff881031affe20 0000000000000046 0000000000000000 ffff8806dec55678 2014-01-07 18:54:45 ffff881031affdb0 ffff880e8c55fd68 0000000000100000 ffff880e8c55fdd8 2014-01-07 18:54:45 ffff881031afdaf8 ffff881031afffd8 000000000000fb88 ffff881031afdaf8 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/14 S 000000000000000e 0 6805 2 0x00000000 2014-01-07 18:54:45 ffff881032187e20 0000000000000046 0000000000000000 ffff8806deccf978 2014-01-07 18:54:45 ffff881032187db0 ffff880e922c82e8 0000000000100000 ffff880e922c8358 2014-01-07 18:54:45 ffff881031afd098 ffff881032187fd8 000000000000fb88 ffff881031afd098 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int/15 S 000000000000000f 0 6806 2 0x00000000 2014-01-07 18:54:45 ffff881030109e20 0000000000000046 0000000000000000 ffff880e86ed5ad8 2014-01-07 18:54:45 ffff881030109db0 ffff880e8cda98c8 0000000000100000 ffff880e8cda9938 2014-01-07 18:54:45 ffff881031afc638 ffff881030109fd8 000000000000fb88 ffff881031afc638 2014-01-07 18:54:45 Call Trace: 2014-01-07 18:54:45 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:45 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:45 [] kthread+0x96/0xa0 2014-01-07 18:54:45 [] child_rip+0xa/0x20 2014-01-07 18:54:45 [] ? kthread+0x0/0xa0 2014-01-07 18:54:45 [] ? child_rip+0x0/0x20 2014-01-07 18:54:45 z_wr_int_h/0 S 0000000000000000 0 6807 2 0x00000000 2014-01-07 18:54:45 ffff8810071e5e20 0000000000000046 ffff8810071e5da0 ffff88103010b538 2014-01-07 18:54:45 0000000000000000 ffff8800446367a8 0000000000000001 ffff880044636740 2014-01-07 18:54:45 ffff88103010bab8 ffff8810071e5fd8 000000000000fb88 ffff88103010bab8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_wr_int_h/1 S 0000000000000001 0 6808 2 0x00000000 2014-01-07 18:54:46 ffff8810318e9e20 0000000000000046 ffff8810318e9da0 ffff88103010aad8 2014-01-07 18:54:46 0000000000000000 ffff8800446367a8 0000000000000001 ffff880044636740 2014-01-07 18:54:46 ffff88103010b058 ffff8810318e9fd8 000000000000fb88 ffff88103010b058 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_wr_int_h/2 S 0000000000000002 0 6809 2 0x00000000 2014-01-07 18:54:46 ffff8810327c1e20 0000000000000046 0000000000000000 ffff88103010a078 2014-01-07 18:54:46 0000000000000000 ffff8800446367a8 0000000000000001 ffff880044636740 2014-01-07 18:54:46 ffff88103010a5f8 ffff8810327c1fd8 000000000000fb88 ffff88103010a5f8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_wr_int_h/3 S 0000000000000003 0 6810 2 0x00000000 2014-01-07 18:54:46 ffff8810327c5e20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:46 0000000000000000 ffff88083436a970 ffff8810327c5de0 ffffffff81051439 2014-01-07 18:54:46 ffff8810327c3af8 ffff8810327c5fd8 000000000000fb88 ffff8810327c3af8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:46 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_wr_int_h/4 S 0000000000000004 0 6811 2 0x00000000 2014-01-07 18:54:46 ffff881030e91e20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:46 0000000000000000 ffff88083436a970 ffff881030e91de0 ffffffff81051439 2014-01-07 18:54:46 ffff8810327c3098 ffff881030e91fd8 000000000000fb88 ffff8810327c3098 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:46 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_0/0 S 0000000000000000 0 6812 2 0x00000000 2014-01-07 18:54:46 ffff881030e93e20 0000000000000046 0000000000000000 ffff880e86ddd7c0 2014-01-07 18:54:46 ffff881030e93de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:46 ffff8810327c2638 ffff881030e93fd8 000000000000fb88 ffff8810327c2638 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_0/1 S 0000000000000001 0 6813 2 0x00000000 2014-01-07 18:54:46 ffff881030e97e20 0000000000000046 ffff88070aaecf00 ffff880e8d2bbd80 2014-01-07 18:54:46 ffff881030e97de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:46 ffff881030e95ab8 ffff881030e97fd8 000000000000fb88 ffff881030e95ab8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_0/2 S 0000000000000002 0 6814 2 0x00000000 2014-01-07 18:54:46 ffff8810318e3e20 0000000000000046 0000000000000000 ffff880e8d6c6940 2014-01-07 18:54:46 ffff8810318e3de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:46 ffff881030e95058 ffff8810318e3fd8 000000000000fb88 ffff881030e95058 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_0/3 S 0000000000000003 0 6815 2 0x00000000 2014-01-07 18:54:46 ffff8810318e5e20 0000000000000046 0000000000000000 ffff880e8cd65c20 2014-01-07 18:54:46 ffff8810318e5de0 ffff88070aaed1d8 0000000000100000 ffff88070aaed218 2014-01-07 18:54:46 ffff881030e945f8 ffff8810318e5fd8 000000000000fb88 ffff881030e945f8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_1/0 S 0000000000000000 0 6816 2 0x00000000 2014-01-07 18:54:46 ffff881030cf3e20 0000000000000046 0000000000000000 ffff881030cf3de4 2014-01-07 18:54:46 ffff881000000000 ffff88083fe81000 ffff880044676740 000000000000000f 2014-01-07 18:54:46 ffff881030cf1af8 ffff881030cf3fd8 000000000000fb88 ffff881030cf1af8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_1/1 S 0000000000000001 0 6817 2 0x00000000 2014-01-07 18:54:46 ffff881030cf7e20 0000000000000046 0000000000000000 ffff881030cf7de4 2014-01-07 18:54:46 ffff881000000000 ffff88083fe81200 ffff880044676740 000000000000000f 2014-01-07 18:54:46 ffff881030cf1098 ffff881030cf7fd8 000000000000fb88 ffff881030cf1098 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_1/2 S 0000000000000002 0 6818 2 0x00000000 2014-01-07 18:54:46 ffff881031979e20 0000000000000046 ffff88070aaecf00 ffff88078c28f660 2014-01-07 18:54:46 ffff881031979de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:46 ffff881030cf0638 ffff881031979fd8 000000000000fb88 ffff881030cf0638 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_1/3 S 0000000000000003 0 6819 2 0x00000000 2014-01-07 18:54:46 ffff88103197de20 0000000000000046 ffff88103197ddb0 ffffffffa0503e9b 2014-01-07 18:54:46 ffff88103197dde0 ffff88074603f660 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:46 ffff88103197bab8 ffff88103197dfd8 000000000000fb88 ffff88103197bab8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_2/0 S 0000000000000000 0 6820 2 0x00000000 2014-01-07 18:54:46 ffff881030c41e20 0000000000000046 0000000000000000 ffff880e8fa4b920 2014-01-07 18:54:46 ffff881030c41de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:46 ffff88103197b058 ffff881030c41fd8 000000000000fb88 ffff88103197b058 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_2/1 S 0000000000000001 0 6821 2 0x00000000 2014-01-07 18:54:46 ffff881030c43e20 0000000000000046 ffff881030c43db0 ffffffffa0503e9b 2014-01-07 18:54:46 ffff881000000000 ffff880615cff200 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:46 ffff88103197a5f8 ffff881030c43fd8 000000000000fb88 ffff88103197a5f8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_2/2 S 0000000000000002 0 6822 2 0x00000000 2014-01-07 18:54:46 ffff881030c47e20 0000000000000046 0000000000000000 ffff881030c47de4 2014-01-07 18:54:46 ffff881000000000 ffff88083fe81400 ffff880044676740 000000000000000f 2014-01-07 18:54:46 ffff881030c45af8 ffff881030c47fd8 000000000000fb88 ffff881030c45af8 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:46 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:46 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:46 [] kthread+0x96/0xa0 2014-01-07 18:54:46 [] child_rip+0xa/0x20 2014-01-07 18:54:46 [] ? kthread+0x0/0xa0 2014-01-07 18:54:46 [] ? child_rip+0x0/0x20 2014-01-07 18:54:46 z_fr_iss_2/3 S 0000000000000003 0 6823 2 0x00000000 2014-01-07 18:54:46 ffff881030935e20 0000000000000046 ffff881030935db0 ffffffffa0503e9b 2014-01-07 18:54:46 ffff881030935de0 ffff880e7cbb6080 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:46 ffff881030c45098 ffff881030935fd8 000000000000fb88 ffff881030c45098 2014-01-07 18:54:46 Call Trace: 2014-01-07 18:54:46 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:46 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_3/0 S 0000000000000000 0 6824 2 0x00000000 2014-01-07 18:54:47 ffff881030937e20 0000000000000046 ffff88070aaecf00 ffff8806dede5360 2014-01-07 18:54:47 ffff881030937de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:47 ffff881030c44638 ffff881030937fd8 000000000000fb88 ffff881030c44638 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_3/1 S 0000000000000001 0 6825 2 0x00000000 2014-01-07 18:54:47 ffff881030d43e20 0000000000000046 0000000000000000 ffff881030d43de4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81200 ffff880044676740 000000000000000f 2014-01-07 18:54:47 ffff881030d41ab8 ffff881030d43fd8 000000000000fb88 ffff881030d41ab8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_3/2 S 0000000000000002 0 6826 2 0x00000000 2014-01-07 18:54:47 ffff881030d47e20 0000000000000046 0000000000000000 ffff881030d47de4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000000f 2014-01-07 18:54:47 ffff881030d41058 ffff881030d47fd8 000000000000fb88 ffff881030d41058 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_3/3 S 0000000000000003 0 6827 2 0x00000000 2014-01-07 18:54:47 ffff8810071e9e20 0000000000000046 ffff88070aaecf00 ffff880e825feda0 2014-01-07 18:54:47 ffff8810071e9de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:47 ffff881030d405f8 ffff8810071e9fd8 000000000000fb88 ffff881030d405f8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_4/0 S 0000000000000000 0 6828 2 0x00000000 2014-01-07 18:54:47 ffff8810071ede20 0000000000000046 0000000000000000 ffff8810071edde4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81000 ffff880044636740 000000000000000f 2014-01-07 18:54:47 ffff8810071ebaf8 ffff8810071edfd8 000000000000fb88 ffff8810071ebaf8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_4/1 S 0000000000000001 0 6829 2 0x00000000 2014-01-07 18:54:47 ffff8810071d1e20 0000000000000046 0000000000000000 ffff8810071d1de4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81200 ffff880044656740 000000000000000f 2014-01-07 18:54:47 ffff8810071eb098 ffff8810071d1fd8 000000000000fb88 ffff8810071eb098 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_4/2 S 0000000000000002 0 6830 2 0x00000000 2014-01-07 18:54:47 ffff8810071d5e20 0000000000000046 ffff8810071d5db0 ffffffffa0503e9b 2014-01-07 18:54:47 ffff8810071d5de0 ffff880e80603060 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:47 ffff8810071ea638 ffff8810071d5fd8 000000000000fb88 ffff8810071ea638 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_4/3 S 0000000000000003 0 6831 2 0x00000000 2014-01-07 18:54:47 ffff8810071d9e20 0000000000000046 ffff8810071d9db0 ffffffffa0503e9b 2014-01-07 18:54:47 ffff881000000000 ffff880e7cd6a7a0 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:47 ffff8810071d7ab8 ffff8810071d9fd8 000000000000fb88 ffff8810071d7ab8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_5/0 S 0000000000000000 0 6832 2 0x00000000 2014-01-07 18:54:47 ffff8810071dde20 0000000000000046 ffff8810071dddb0 ffffffffa0503e9b 2014-01-07 18:54:47 ffff8810071ddde0 ffff880e8eb411c0 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:47 ffff8810071d7058 ffff8810071ddfd8 000000000000fb88 ffff8810071d7058 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_5/1 S 0000000000000001 0 6833 2 0x00000000 2014-01-07 18:54:47 ffff8810071dfe20 0000000000000046 ffff88070aaecf00 ffff880e8eb034c0 2014-01-07 18:54:47 ffff8810071dfde0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:47 ffff8810071d65f8 ffff8810071dffd8 000000000000fb88 ffff8810071d65f8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_5/2 S 0000000000000002 0 6834 2 0x00000000 2014-01-07 18:54:47 ffff8810071f3e20 0000000000000046 0000000000000000 ffff8810071f3de4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000000f 2014-01-07 18:54:47 ffff8810071f1af8 ffff8810071f3fd8 000000000000fb88 ffff8810071f1af8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_5/3 S 0000000000000003 0 6835 2 0x00000000 2014-01-07 18:54:47 ffff8810071f7e20 0000000000000046 0000000000000000 ffff8810071f7de4 2014-01-07 18:54:47 ffff881000000000 ffff88083fe81600 ffff880044616740 000000000000000f 2014-01-07 18:54:47 ffff8810071f1098 ffff8810071f7fd8 000000000000fb88 ffff8810071f1098 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_6/0 S 0000000000000000 0 6836 2 0x00000000 2014-01-07 18:54:47 ffff8810071f9e20 0000000000000046 ffff88070aaecf00 ffff880e8d5911c0 2014-01-07 18:54:47 ffff8810071f9de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:47 ffff8810071f0638 ffff8810071f9fd8 000000000000fb88 ffff8810071f0638 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_6/1 S 0000000000000001 0 6837 2 0x00000000 2014-01-07 18:54:47 ffff8810071ffe20 0000000000000046 ffff8810071ffdb0 ffffffffa0503e9b 2014-01-07 18:54:47 ffff8810071ffde0 ffff880e8cd65c20 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:47 ffff8810071fdab8 ffff8810071fffd8 000000000000fb88 ffff8810071fdab8 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:47 z_fr_iss_6/2 S 0000000000000002 0 6838 2 0x00000000 2014-01-07 18:54:47 ffff881007203e20 0000000000000046 0000000000000000 ffffffffa0503e9b 2014-01-07 18:54:47 ffff881007203de0 ffff8806dede5360 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:47 ffff8810071fd058 ffff881007203fd8 000000000000fb88 ffff8810071fd058 2014-01-07 18:54:47 Call Trace: 2014-01-07 18:54:47 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:47 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:47 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:47 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:47 [] kthread+0x96/0xa0 2014-01-07 18:54:47 [] child_rip+0xa/0x20 2014-01-07 18:54:47 [] ? kthread+0x0/0xa0 2014-01-07 18:54:47 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_iss_6/3 S 0000000000000003 0 6839 2 0x00000000 2014-01-07 18:54:48 ffff881007205e20 0000000000000046 ffff88070aaecf00 ffff88074603f660 2014-01-07 18:54:48 ffff881007205de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:48 ffff8810071fc5f8 ffff881007205fd8 000000000000fb88 ffff8810071fc5f8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_iss_7/0 S 0000000000000000 0 6840 2 0x00000000 2014-01-07 18:54:48 ffff881007209e20 0000000000000046 ffff88070aaecf00 ffff880e80603060 2014-01-07 18:54:48 ffff881007209de0 ffff88070aaed1d8 0000000000000003 ffff88070aaed218 2014-01-07 18:54:48 ffff881007207af8 ffff881007209fd8 000000000000fb88 ffff881007207af8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_iss_7/1 S 0000000000000001 0 6841 2 0x00000000 2014-01-07 18:54:48 ffff88100720de20 0000000000000046 ffff88100720ddb0 ffffffffa0503e9b 2014-01-07 18:54:48 ffff88100720dde0 ffff880e825f8d60 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:48 ffff881007207098 ffff88100720dfd8 000000000000fb88 ffff881007207098 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_iss_7/2 S 0000000000000002 0 6842 2 0x00000000 2014-01-07 18:54:48 ffff88100720fe20 0000000000000046 0000000000000000 ffff88100720fde4 2014-01-07 18:54:48 ffff881000000000 ffff88083fe81400 ffff880044616740 000000000000000f 2014-01-07 18:54:48 ffff881007206638 ffff88100720ffd8 000000000000fb88 ffff881007206638 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_iss_7/3 S 0000000000000003 0 6843 2 0x00000000 2014-01-07 18:54:48 ffff881007213e20 0000000000000046 ffff881007213db0 ffffffffa0503e9b 2014-01-07 18:54:48 ffff881000000000 ffff880e8d2bbd80 0000000000000008 ffff8807d2d1c000 2014-01-07 18:54:48 ffff881007211ab8 ffff881007213fd8 000000000000fb88 ffff881007211ab8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? spa_taskq_dispatch_ent+0x6b/0xa0 [zfs] 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_fr_int/0 S 0000000000000000 0 6844 2 0x00000000 2014-01-07 18:54:48 ffff881007219e20 0000000000000046 ffff881007219d90 ffffffff81096db6 2014-01-07 18:54:48 0000000000000000 ffff8808342a4630 ffff881007219de0 ffffffff81051439 2014-01-07 18:54:48 ffff881007211058 ffff881007219fd8 000000000000fb88 ffff881007211058 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:48 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_cl_iss/0 S 0000000000000000 0 6845 2 0x00000000 2014-01-07 18:54:48 ffff88100721be20 0000000000000046 ffff88100721bd90 ffffffff81096db6 2014-01-07 18:54:48 0000000000000000 ffff880833f7ff30 ffff88100721bde0 ffffffff81051439 2014-01-07 18:54:48 ffff8810072105f8 ffff88100721bfd8 000000000000fb88 ffff8810072105f8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:48 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_cl_int/0 S 0000000000000000 0 6846 2 0x00000000 2014-01-07 18:54:48 ffff88100721fe20 0000000000000046 ffff88100721fd90 ffffffff81096db6 2014-01-07 18:54:48 0000000000000000 ffff880833e40630 ffff88100721fde0 ffffffff81051439 2014-01-07 18:54:48 ffff88100721daf8 ffff88100721ffd8 000000000000fb88 ffff88100721daf8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:48 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_ioctl_iss/0 S 0000000000000000 0 6847 2 0x00000000 2014-01-07 18:54:48 ffff881007223e20 0000000000000046 ffff881007223da0 ffffffffa0519896 2014-01-07 18:54:48 ffff881007223de0 ffff880e7d53b060 ffff880815181000 0000000000000001 2014-01-07 18:54:48 ffff88100721d098 ffff881007223fd8 000000000000fb88 ffff88100721d098 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? vdev_disk_io_start+0x146/0x1a0 [zfs] 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 z_ioctl_int/0 S 0000000000000000 0 6848 2 0x00000000 2014-01-07 18:54:48 ffff881007225e20 0000000000000046 0000000000000000 ffff880e7d53b378 2014-01-07 18:54:48 ffff881007225db0 ffff88070aaed1a8 0000000000100000 ffff88070aaed218 2014-01-07 18:54:48 ffff88100721c638 ffff881007225fd8 000000000000fb88 ffff88100721c638 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 zfs_iput_task S 0000000000000000 0 6855 2 0x00000000 2014-01-07 18:54:48 ffff88100722de20 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:48 0000000000000000 ffff8807d2d7f130 ffff88100722dde0 ffffffff81051439 2014-01-07 18:54:48 ffff881007229af8 ffff88100722dfd8 000000000000fb88 ffff881007229af8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:48 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:48 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:48 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 txg_quiesce S 0000000000000000 0 6856 2 0x00000000 2014-01-07 18:54:48 ffff881007233d70 0000000000000046 0000000000000000 ffff881007233d34 2014-01-07 18:54:48 0000000200000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:48 ffff881007229098 ffff881007233fd8 000000000000fb88 ffff881007229098 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] cv_wait_common+0xed/0x100 [spl] 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:48 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:48 [] __cv_wait_interruptible+0x15/0x20 [spl] 2014-01-07 18:54:48 [] txg_thread_wait+0x1d/0x40 [zfs] 2014-01-07 18:54:48 [] txg_quiesce_thread+0x2bc/0x2d0 [zfs] 2014-01-07 18:54:48 [] ? txg_quiesce_thread+0x0/0x2d0 [zfs] 2014-01-07 18:54:48 [] thread_generic_wrapper+0x5f/0x70 [spl] 2014-01-07 18:54:48 [] ? thread_generic_wrapper+0x0/0x70 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 txg_sync S 0000000000000002 0 6857 2 0x00000000 2014-01-07 18:54:48 ffff881007235cb0 0000000000000046 0000000000000000 ffff881007235c74 2014-01-07 18:54:48 ffffffff00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:48 ffff881007228638 ffff881007235fd8 000000000000fb88 ffff881007228638 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:48 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:48 [] __cv_timedwait_common+0x9f/0x130 [spl] 2014-01-07 18:54:48 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:48 [] __cv_timedwait_interruptible+0x13/0x20 [spl] 2014-01-07 18:54:48 [] txg_thread_wait+0x34/0x40 [zfs] 2014-01-07 18:54:48 [] txg_sync_thread+0x12d/0x4b0 [zfs] 2014-01-07 18:54:48 [] ? txg_sync_thread+0x0/0x4b0 [zfs] 2014-01-07 18:54:48 [] thread_generic_wrapper+0x5f/0x70 [spl] 2014-01-07 18:54:48 [] ? thread_generic_wrapper+0x0/0x70 [spl] 2014-01-07 18:54:48 [] kthread+0x96/0xa0 2014-01-07 18:54:48 [] child_rip+0xa/0x20 2014-01-07 18:54:48 [] ? kthread+0x0/0xa0 2014-01-07 18:54:48 [] ? child_rip+0x0/0x20 2014-01-07 18:54:48 obd_zombid S 0000000000000001 0 7040 2 0x00000000 2014-01-07 18:54:48 ffff881030d71e60 0000000000000046 ffff881030915500 ffff881030d71e80 2014-01-07 18:54:48 ffff881030915500 ffff881030915500 ffff881030d71e30 ffffffffa03847b1 2014-01-07 18:54:48 ffff881030915ab8 ffff881030d71fd8 000000000000fb88 ffff881030915ab8 2014-01-07 18:54:48 Call Trace: 2014-01-07 18:54:48 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:48 [] obd_zombie_impexp_thread+0xd9/0x190 [obdclass] 2014-01-07 18:54:48 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? obd_zombie_impexp_thread+0x0/0x190 [obdclass] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr00_0 S 0000000000000001 0 7041 2 0x00000000 2014-01-07 18:54:49 ffff881030d73de0 0000000000000046 0000000000000000 0000000a00000003 2014-01-07 18:54:49 ffff881030d73d50 ffff88100723bcf8 0000000000000000 ffff88100723b500 2014-01-07 18:54:49 ffff88100723bab8 ffff881030d73fd8 000000000000fb88 ffff88100723bab8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr00_0 S 0000000000000001 0 7042 2 0x00000000 2014-01-07 18:54:49 ffff8810072f9de0 0000000000000046 0000000000000000 0000000a00000003 2014-01-07 18:54:49 ffff8810072f9d50 ffff881007273d38 0000000000000000 ffff881007273540 2014-01-07 18:54:49 ffff881007273af8 ffff8810072f9fd8 000000000000fb88 ffff881007273af8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr00_0 S 0000000000000001 0 7043 2 0x00000000 2014-01-07 18:54:49 ffff8810072ffde0 0000000000000046 0000000000000000 0000000a00000003 2014-01-07 18:54:49 ffff8810072ffd50 ffff8810072732d8 0000000000000000 ffff881007272ae0 2014-01-07 18:54:49 ffff881007273098 ffff8810072fffd8 000000000000fb88 ffff881007273098 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr00_0 S 0000000000000001 0 7044 2 0x00000000 2014-01-07 18:54:49 ffff8810072c1de0 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:49 ffff8810072c1d50 ffffffffa0963398 ffff8810072c1da0 ffffffff81051439 2014-01-07 18:54:49 ffff88100723a5f8 ffff8810072c1fd8 000000000000fb88 ffff88100723a5f8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:49 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr01_0 S 0000000000000004 0 7045 2 0x00000000 2014-01-07 18:54:49 ffff8810072c3de0 0000000000000046 0000000000000000 ffff8810072c3d30 2014-01-07 18:54:49 ffff8810072c3d50 ffff8810321cdcf8 0000000000000000 ffff8810321cd500 2014-01-07 18:54:49 ffff8810321cdab8 ffff8810072c3fd8 000000000000fb88 ffff8810321cdab8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr01_0 S 0000000000000004 0 7046 2 0x00000000 2014-01-07 18:54:49 ffff8810072c7de0 0000000000000046 0000000000000000 ffff8810072c7d30 2014-01-07 18:54:49 ffff8810072c7d50 ffff8810300e3298 0000000000000000 ffff8810300e2aa0 2014-01-07 18:54:49 ffff8810300e3058 ffff8810072c7fd8 000000000000fb88 ffff8810300e3058 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr01_0 S 0000000000000004 0 7047 2 0x00000000 2014-01-07 18:54:49 ffff8810072c9de0 0000000000000046 0000000000000000 ffff8810072c9d30 2014-01-07 18:54:49 ffff8810072c9d50 ffff8810301f7d38 0000000000000000 ffff8810301f7540 2014-01-07 18:54:49 ffff8810301f7af8 ffff8810072c9fd8 000000000000fb88 ffff8810301f7af8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr01_0 S 0000000000000004 0 7048 2 0x00000000 2014-01-07 18:54:49 ffff8810072cbde0 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:49 ffff8810072cbd50 ffffffffa0963398 ffff8810072cbda0 ffffffff81051439 2014-01-07 18:54:49 ffff881032633098 ffff8810072cbfd8 000000000000fb88 ffff881032633098 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:49 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr02_0 S 0000000000000008 0 7049 2 0x00000000 2014-01-07 18:54:49 ffff8810072cfde0 0000000000000046 0000000000000000 ffff8810072cfd30 2014-01-07 18:54:49 ffff8810072cfd50 ffff881031c15d38 0000000000000000 ffff881031c15540 2014-01-07 18:54:49 ffff881031c15af8 ffff8810072cffd8 000000000000fb88 ffff881031c15af8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr02_0 S 0000000000000008 0 7050 2 0x00000000 2014-01-07 18:54:49 ffff881007303de0 0000000000000046 0000000000000000 ffff881007303d30 2014-01-07 18:54:49 ffff881007303d50 ffff8810332372d8 0000000000000000 ffff881033236ae0 2014-01-07 18:54:49 ffff881033237098 ffff881007303fd8 000000000000fb88 ffff881033237098 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr02_0 S 0000000000000008 0 7051 2 0x00000000 2014-01-07 18:54:49 ffff881007305de0 0000000000000046 0000000000000000 ffff881007305d30 2014-01-07 18:54:49 ffff881007305d50 ffff881033236878 0000000000000000 ffff881033236080 2014-01-07 18:54:49 ffff881033236638 ffff881007305fd8 000000000000fb88 ffff881033236638 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr02_0 S 0000000000000008 0 7052 2 0x00000000 2014-01-07 18:54:49 ffff881007309de0 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:49 ffff881007309d50 ffffffffa0963398 ffff881007309da0 ffffffff81051439 2014-01-07 18:54:49 ffff881031e91058 ffff881007309fd8 000000000000fb88 ffff881031e91058 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:49 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr03_0 S 000000000000000c 0 7053 2 0x00000000 2014-01-07 18:54:49 ffff88100730bde0 0000000000000046 0000000000000000 ffff88100730bd30 2014-01-07 18:54:49 ffff88100730bd50 ffff881030062878 0000000000000000 ffff881030062080 2014-01-07 18:54:49 ffff881030062638 ffff88100730bfd8 000000000000fb88 ffff881030062638 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr03_0 S 000000000000000c 0 7054 2 0x00000000 2014-01-07 18:54:49 ffff88100730fde0 0000000000000046 0000000000000000 ffff88100730fd30 2014-01-07 18:54:49 ffff88100730fd50 ffff88100730dd38 0000000000000000 ffff88100730d540 2014-01-07 18:54:49 ffff88100730daf8 ffff88100730ffd8 000000000000fb88 ffff88100730daf8 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:49 [] child_rip+0xa/0x20 2014-01-07 18:54:49 [] ? kthread+0x0/0xa0 2014-01-07 18:54:49 [] ? child_rip+0x0/0x20 2014-01-07 18:54:49 ptlrpc_hr03_0 S 000000000000000c 0 7055 2 0x00000000 2014-01-07 18:54:49 ffff881007313de0 0000000000000046 0000000000000000 ffff881007313d30 2014-01-07 18:54:49 ffff881007313d50 ffff88100730d2d8 0000000000000000 ffff88100730cae0 2014-01-07 18:54:49 ffff88100730d098 ffff881007313fd8 000000000000fb88 ffff88100730d098 2014-01-07 18:54:49 Call Trace: 2014-01-07 18:54:49 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:49 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:49 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:49 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpc_hr03_0 S 000000000000000c 0 7056 2 0x00000000 2014-01-07 18:54:50 ffff881007315de0 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:50 ffff881007315d50 ffffffffa0963398 ffff881007315da0 ffffffff81051439 2014-01-07 18:54:50 ffff88100730c638 ffff881007315fd8 000000000000fb88 ffff88100730c638 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:50 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:50 [] ptlrpc_hr_main+0x14a/0x750 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpc_hr_main+0x0/0x750 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_rcv S 0000000000000008 0 7057 2 0x00000000 2014-01-07 18:54:50 ffff88100731bd50 0000000000000046 0000000000000000 ffff88100731bd14 2014-01-07 18:54:50 ffff881000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:50 ffff881007319ab8 ffff88100731bfd8 000000000000fb88 ffff881007319ab8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_0 S 000000000000000a 0 7058 2 0x00000000 2014-01-07 18:54:50 ffff88100731fd50 0000000000000046 0000000000000000 ffff88100731fd90 2014-01-07 18:54:50 ffff88100731fd50 0000082000000000 0000000000000010 ffff88100731fdb0 2014-01-07 18:54:50 ffff881007319058 ffff88100731ffd8 000000000000fb88 ffff881007319058 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_1 S 0000000000000003 0 7059 2 0x00000000 2014-01-07 18:54:50 ffff881007321d50 0000000000000046 0000000000000000 ffff881007321d14 2014-01-07 18:54:50 ffff881000000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:50 ffff8810073185f8 ffff881007321fd8 000000000000fb88 ffff8810073185f8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_2 S 000000000000000c 0 7060 2 0x00000000 2014-01-07 18:54:50 ffff881007325d50 0000000000000046 0000000000000000 ffff881007325d14 2014-01-07 18:54:50 ffff881000000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:54:50 ffff881007323af8 ffff881007325fd8 000000000000fb88 ffff881007323af8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_3 S 0000000000000004 0 7061 2 0x00000000 2014-01-07 18:54:50 ffff881007329d50 0000000000000046 0000000000000000 ffff881007329d14 2014-01-07 18:54:50 ffff881000000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:50 ffff881007323098 ffff881007329fd8 000000000000fb88 ffff881007323098 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_4 S 000000000000000e 0 7062 2 0x00000000 2014-01-07 18:54:50 ffff88100732bd50 0000000000000046 ffff88100732bd18 ffff88100732bd14 2014-01-07 18:54:50 ffff88100732bd50 ffff88083fe82c00 ffff880044636740 0000000000000407 2014-01-07 18:54:50 ffff881007322638 ffff88100732bfd8 000000000000fb88 ffff881007322638 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_5 S 0000000000000000 0 7063 2 0x00000000 2014-01-07 18:54:50 ffff88100732fd50 0000000000000046 0000000000000000 ffff88100732fd14 2014-01-07 18:54:50 ffff881000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:50 ffff88100732dab8 ffff88100732ffd8 000000000000fb88 ffff88100732dab8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_6 S 0000000000000009 0 7064 2 0x00000000 2014-01-07 18:54:50 ffff881007335d50 0000000000000046 0000000000000028 ffff881007335d90 2014-01-07 18:54:50 ffff881007335d50 0000082000000000 0000000000000010 ffff881007335db0 2014-01-07 18:54:50 ffff88100732d058 ffff881007335fd8 000000000000fb88 ffff88100732d058 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_7 S 0000000000000003 0 7065 2 0x00000000 2014-01-07 18:54:50 ffff881007337d50 0000000000000046 0000000000000028 ffff881007337d90 2014-01-07 18:54:50 ffff881007337d50 0000082000000000 0000000000000010 ffff881007337db0 2014-01-07 18:54:50 ffff88100732c5f8 ffff881007337fd8 000000000000fb88 ffff88100732c5f8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_8 S 0000000000000009 0 7066 2 0x00000000 2014-01-07 18:54:50 ffff88100733bd50 0000000000000046 ffff88100733bd18 ffff88100733bd14 2014-01-07 18:54:50 ffff88100733bd50 ffff88083fe82200 ffff880044636740 0000000000000607 2014-01-07 18:54:50 ffff881007339af8 ffff88100733bfd8 000000000000fb88 ffff881007339af8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_9 S 0000000000000009 0 7067 2 0x00000000 2014-01-07 18:54:50 ffff88100733fd50 0000000000000046 0000000000000028 ffff88100733fd90 2014-01-07 18:54:50 ffff88100733fd50 0000082000000000 0000000000000010 ffff88100733fdb0 2014-01-07 18:54:50 ffff881007339098 ffff88100733ffd8 000000000000fb88 ffff881007339098 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_10 S 0000000000000000 0 7068 2 0x00000000 2014-01-07 18:54:50 ffff881007341d50 0000000000000046 0000000000000028 ffff881007341d90 2014-01-07 18:54:50 ffff881007341d50 0000082000000000 0000000000000010 ffff881007341db0 2014-01-07 18:54:50 ffff881007338638 ffff881007341fd8 000000000000fb88 ffff881007338638 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_11 S 000000000000000b 0 7069 2 0x00000000 2014-01-07 18:54:50 ffff881007345d50 0000000000000046 ffff881007345d18 ffff881007345d14 2014-01-07 18:54:50 ffff881007345d50 ffff88083fe82600 ffff880044636740 0000000000000607 2014-01-07 18:54:50 ffff881007343ab8 ffff881007345fd8 000000000000fb88 ffff881007343ab8 2014-01-07 18:54:50 Call Trace: 2014-01-07 18:54:50 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:50 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:50 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:50 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:50 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:50 [] kthread+0x96/0xa0 2014-01-07 18:54:50 [] child_rip+0xa/0x20 2014-01-07 18:54:50 [] ? kthread+0x0/0xa0 2014-01-07 18:54:50 [] ? child_rip+0x0/0x20 2014-01-07 18:54:50 ptlrpcd_12 S 0000000000000001 0 7070 2 0x00000000 2014-01-07 18:54:50 ffff881007349d50 0000000000000046 0000000000000028 ffff881007349d90 2014-01-07 18:54:50 ffff881007349d50 0000082000000000 0000000000000010 ffff881007349db0 2014-01-07 18:54:51 ffff881007343058 ffff881007349fd8 000000000000fb88 ffff881007343058 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ptlrpcd_13 S 0000000000000008 0 7071 2 0x00000000 2014-01-07 18:54:51 ffff88100734dd50 0000000000000046 0000000000000028 ffff88100734dd90 2014-01-07 18:54:51 ffff88100734dd50 0000082000000000 0000000000000010 ffff88100734ddb0 2014-01-07 18:54:51 ffff8810073425f8 ffff88100734dfd8 000000000000fb88 ffff8810073425f8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ptlrpcd_14 S 0000000000000001 0 7072 2 0x00000000 2014-01-07 18:54:51 ffff881007351d50 0000000000000046 0000000000000000 ffff881007351d90 2014-01-07 18:54:51 ffff881007351d50 0000082000000000 0000000000000010 ffff881007351db0 2014-01-07 18:54:51 ffff88100734faf8 ffff881007351fd8 000000000000fb88 ffff88100734faf8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ptlrpcd_15 S 000000000000000a 0 7073 2 0x00000000 2014-01-07 18:54:51 ffff881007355d50 0000000000000046 0000000000000028 ffff881007355d90 2014-01-07 18:54:51 ffff881007355d50 0000082000000000 0000000000000010 ffff881007355db0 2014-01-07 18:54:51 ffff88100734f098 ffff881007355fd8 000000000000fb88 ffff88100734f098 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] ptlrpcd+0x1ec/0x370 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? ptlrpcd+0x0/0x370 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ll_ping S 0000000000000001 0 7074 2 0x00000000 2014-01-07 18:54:51 ffff881007357d50 0000000000000046 ffff881007357db0 0000000007357d90 2014-01-07 18:54:51 0000000000000000 ffff8807bebb1c1c 0000004c00000286 ffffffffa090ccf4 2014-01-07 18:54:51 ffff88100734e638 ffff881007357fd8 000000000000fb88 ffff88100734e638 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] ptlrpc_pinger_main+0x736/0x820 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? ptlrpc_pinger_main+0x0/0x820 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 sptlrpc_gc S 0000000000000001 0 7075 2 0x00000000 2014-01-07 18:54:51 ffff88100735bda0 0000000000000046 ffff880044656740 0000000000000003 2014-01-07 18:54:51 ffff88100735bd30 0000000000000286 ffff88100735bd40 ffffffff810810cc 2014-01-07 18:54:51 ffff881007359ab8 ffff88100735bfd8 000000000000fb88 ffff881007359ab8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:51 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:51 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:51 [] sec_gc_main+0x277/0x380 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? sec_gc_main+0x0/0x380 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ll_capa S 0000000000000001 0 7076 2 0x00000000 2014-01-07 18:54:51 ffff88100735fe50 0000000000000046 0000000000000000 ffffffff81096db6 2014-01-07 18:54:51 ffff88100735fe10 ffffffffa0c41088 ffff88100735fe10 ffffffff81051439 2014-01-07 18:54:51 ffff881007359058 ffff88100735ffd8 000000000000fb88 ffff881007359058 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:51 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:51 [] capa_thread_main+0x9c3/0xd50 [lustre] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? capa_thread_main+0x0/0xd50 [lustre] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb00_000 S 0000000000000000 0 7081 2 0x00000000 2014-01-07 18:54:51 ffff881007371da0 0000000000000046 0000000000000000 ffff880044636740 2014-01-07 18:54:51 0000000000000000 0000000000000001 ffff881007371d40 ffffffff81065d54 2014-01-07 18:54:51 ffff8810073585f8 ffff881007371fd8 000000000000fb88 ffff8810073585f8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:51 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 lc_watchdogd S 0000000000000001 0 7082 2 0x00000000 2014-01-07 18:54:51 ffff881007377e20 0000000000000046 0000000000000000 ffffffffa03cf4f0 2014-01-07 18:54:51 0000000000000000 0000000000000000 ffff881007377da0 ffffffff81063422 2014-01-07 18:54:51 ffff881007375ab8 ffff881007377fd8 000000000000fb88 ffff881007375ab8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? default_wake_function+0x12/0x20 2014-01-07 18:54:51 [] lcw_dispatch_main+0xc9/0x930 [libcfs] 2014-01-07 18:54:51 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:51 [] ? lcw_dispatch_main+0x0/0x930 [libcfs] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb00_001 S 0000000000000000 0 7083 2 0x00000000 2014-01-07 18:54:51 ffff88100737bda0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:51 ffff88100737bd30 ffff88083239f540 ffff88100737bd30 ffffffff8105231d 2014-01-07 18:54:51 ffff881007375058 ffff88100737bfd8 000000000000fb88 ffff881007375058 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:51 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:51 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb01_000 S 0000000000000005 0 7084 2 0x00000000 2014-01-07 18:54:51 ffff88100737dda0 0000000000000046 0000000000000010 ffff8807d1f86680 2014-01-07 18:54:51 ffff8807d16c0400 0000000000000000 0000000000000002 ffff8807d1a36380 2014-01-07 18:54:51 ffff8810073745f8 ffff88100737dfd8 000000000000fb88 ffff8810073745f8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb01_001 S 0000000000000007 0 7085 2 0x00000000 2014-01-07 18:54:51 ffff881007381da0 0000000000000046 0000000000000000 ffff8807d1f86440 2014-01-07 18:54:51 ffff8807d16c0400 0000000000000000 0000000000000002 ffff8807d1a36380 2014-01-07 18:54:51 ffff88100737faf8 ffff881007381fd8 000000000000fb88 ffff88100737faf8 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb02_000 S 0000000000000008 0 7086 2 0x00000000 2014-01-07 18:54:51 ffff881007385da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:51 ffff881007385d30 ffff88083239f540 ffff881007385d30 ffffffff8105231d 2014-01-07 18:54:51 ffff88100737f098 ffff881007385fd8 000000000000fb88 ffff88100737f098 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:51 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:51 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:51 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:51 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:51 [] kthread+0x96/0xa0 2014-01-07 18:54:51 [] child_rip+0xa/0x20 2014-01-07 18:54:51 [] ? kthread+0x0/0xa0 2014-01-07 18:54:51 [] ? child_rip+0x0/0x20 2014-01-07 18:54:51 ldlm_cb02_001 S 0000000000000008 0 7087 2 0x00000000 2014-01-07 18:54:51 ffff881007387da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:51 ffff881007387d30 ffff88083239f540 ffff881007387d30 ffffffff8105231d 2014-01-07 18:54:51 ffff88100737e638 ffff881007387fd8 000000000000fb88 ffff88100737e638 2014-01-07 18:54:51 Call Trace: 2014-01-07 18:54:51 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:51 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:51 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cb03_000 S 000000000000000c 0 7088 2 0x00000000 2014-01-07 18:54:52 ffff88100738bda0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:52 ffff88100738bd30 ffff88083239f540 ffff88100738bd30 ffffffff8105231d 2014-01-07 18:54:52 ffff881007389ab8 ffff88100738bfd8 000000000000fb88 ffff881007389ab8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:52 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cb03_001 S 000000000000000c 0 7089 2 0x00000000 2014-01-07 18:54:52 ffff881007391da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:52 ffff881007391d30 ffff88083239f540 ffff881007391d30 ffffffff8105231d 2014-01-07 18:54:52 ffff881007389058 ffff881007391fd8 000000000000fb88 ffff881007389058 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:52 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn00_000 S 0000000000000002 0 7090 2 0x00000000 2014-01-07 18:54:52 ffff881006c9bda0 0000000000000046 0000000000000000 ffff8807d2732680 2014-01-07 18:54:52 ffff8807d16c0000 0000000000000000 0000000000000058 ffff8807d1a35dc0 2014-01-07 18:54:52 ffff8810073885f8 ffff881006c9bfd8 000000000000fb88 ffff8810073885f8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:52 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn00_001 S 0000000000000003 0 7091 2 0x00000000 2014-01-07 18:54:52 ffff881006c9fda0 0000000000000046 0000000000000000 ffff881006c9fd64 2014-01-07 18:54:52 ffff8807d16c0000 ffff88083fe81400 ffff880044616740 000000000000038d 2014-01-07 18:54:52 ffff881006c9daf8 ffff881006c9ffd8 000000000000fb88 ffff881006c9daf8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn00_002 S 0000000000000000 0 7092 2 0x00000000 2014-01-07 18:54:52 ffff881006ca3da0 0000000000000046 0000000000000000 ffff881006ca3d64 2014-01-07 18:54:52 ffff880700000000 ffff88083fe81000 ffff88085c4d6740 000000000000ad5a 2014-01-07 18:54:52 ffff881006c9d098 ffff881006ca3fd8 000000000000fb88 ffff881006c9d098 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn01_000 S 0000000000000004 0 7093 2 0x00000000 2014-01-07 18:54:52 ffff881006ca5da0 0000000000000046 0000000000000000 ffff881006ca5d64 2014-01-07 18:54:52 ffff8807d16ffc00 ffff88083fe81a00 ffff8800446f6740 0000000000000400 2014-01-07 18:54:52 ffff881006c9c638 ffff881006ca5fd8 000000000000fb88 ffff881006c9c638 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn01_001 S 0000000000000004 0 7094 2 0x00000000 2014-01-07 18:54:52 ffff881006ca9da0 0000000000000046 0000000000000010 ffff8807d060cf00 2014-01-07 18:54:52 ffff8807d16ffc00 0000000000000000 0000000000000023 ffff8807d1a35dc0 2014-01-07 18:54:52 ffff881006ca7ab8 ffff881006ca9fd8 000000000000fb88 ffff881006ca7ab8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:52 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn01_002 S 0000000000000005 0 7095 2 0x00000000 2014-01-07 18:54:52 ffff881006cadda0 0000000000000046 0000000000000000 ffff8807d060ccc0 2014-01-07 18:54:52 ffff8807d16ffc00 0000000000000000 000000000000005d ffff8807d1a35dc0 2014-01-07 18:54:52 ffff881006ca7058 ffff881006cadfd8 000000000000fb88 ffff881006ca7058 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:52 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn02_000 S 0000000000000008 0 7096 2 0x00000000 2014-01-07 18:54:52 ffff881006cb1da0 0000000000000046 ffff881006cb1d68 ffff881006cb1d64 2014-01-07 18:54:52 ffff881030b29400 ffff88083fe82000 ffff88085c456740 0000000000000400 2014-01-07 18:54:52 ffff881006ca65f8 ffff881006cb1fd8 000000000000fb88 ffff881006ca65f8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn02_001 S 000000000000000b 0 7097 2 0x00000000 2014-01-07 18:54:52 ffff881006cb5da0 0000000000000046 0000000000000000 ffff8810322473c0 2014-01-07 18:54:52 ffff881030b29400 0000000000000000 000000000000004b ffff8807d1a35dc0 2014-01-07 18:54:52 ffff881006cb3af8 ffff881006cb5fd8 000000000000fb88 ffff881006cb3af8 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:52 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn02_002 S 0000000000000009 0 7098 2 0x00000000 2014-01-07 18:54:52 ffff881006cb9da0 0000000000000046 0000000000000000 ffff881006cb9d64 2014-01-07 18:54:52 ffff881000000000 ffff88083fe82200 ffff88085c476740 0000000000000400 2014-01-07 18:54:52 ffff881006cb3098 ffff881006cb9fd8 000000000000fb88 ffff881006cb3098 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn03_000 S 000000000000000f 0 7099 2 0x00000000 2014-01-07 18:54:52 ffff881006cbbda0 0000000000000046 0000000000000010 ffff881031c64380 2014-01-07 18:54:52 ffff881030b29800 0000000000000000 0000000000000000 ffff8807d1a35dc0 2014-01-07 18:54:52 ffff881006cb2638 ffff881006cbbfd8 000000000000fb88 ffff881006cb2638 2014-01-07 18:54:52 Call Trace: 2014-01-07 18:54:52 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:52 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:52 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:52 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:52 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:52 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:52 [] kthread+0x96/0xa0 2014-01-07 18:54:52 [] child_rip+0xa/0x20 2014-01-07 18:54:52 [] ? kthread+0x0/0xa0 2014-01-07 18:54:52 [] ? child_rip+0x0/0x20 2014-01-07 18:54:52 ldlm_cn03_001 S 000000000000000c 0 7100 2 0x00000000 2014-01-07 18:54:52 ffff881006cbfda0 0000000000000046 0000000000000000 ffff881006cbfd64 2014-01-07 18:54:52 ffff881000000000 ffff88083fe82800 ffff8800446f6740 0000000000000400 2014-01-07 18:54:52 ffff881006cbdab8 ffff881006cbffd8 000000000000fb88 ffff881006cbdab8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ldlm_cn03_002 S 000000000000000f 0 7101 2 0x00000000 2014-01-07 18:54:53 ffff881006cc3da0 0000000000000046 0000000000000000 ffff88102f561740 2014-01-07 18:54:53 ffff881030b29800 0000000000000000 0000000000000027 ffff8807d1a35dc0 2014-01-07 18:54:53 ffff881006cbd058 ffff881006cc3fd8 000000000000fb88 ffff881006cbd058 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:53 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ldlm_bl_00 S 0000000000000000 0 7102 2 0x00000000 2014-01-07 18:54:53 ffff881006cc5e40 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:53 0000000000000002 0000000000000000 ffff881006cc5e10 ffff8800b8775740 2014-01-07 18:54:53 ffff881006cbc5f8 ffff881006cc5fd8 000000000000fb88 ffff881006cbc5f8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ldlm_bl_thread_main+0x345/0x3c0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ? ldlm_bl_thread_main+0x0/0x3c0 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ldlm_bl_01 S 0000000000000002 0 7103 2 0x00000000 2014-01-07 18:54:53 ffff881006ccbe40 0000000000000046 0000000000000000 ffff881006ccbe04 2014-01-07 18:54:53 0000000000000000 ffff88083fe81400 ffff880044696740 0000000000000400 2014-01-07 18:54:53 ffff881006cc9af8 ffff881006ccbfd8 000000000000fb88 ffff881006cc9af8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ldlm_bl_thread_main+0x345/0x3c0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ? ldlm_bl_thread_main+0x0/0x3c0 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ldlm_elt S 0000000000000001 0 7104 2 0x00000000 2014-01-07 18:54:53 ffff881006ccfe50 0000000000000046 ffff881006ccfdc0 ffffffff81096db6 2014-01-07 18:54:53 ffff881006ccfe10 ffffffffa095b850 ffff881006ccfe10 ffffffff81051439 2014-01-07 18:54:53 ffff881006cc9098 ffff881006ccffd8 000000000000fb88 ffff881006cc9098 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ? autoremove_wake_function+0x16/0x40 2014-01-07 18:54:53 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:54:53 [] expired_lock_main+0x269/0x860 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ? expired_lock_main+0x0/0x860 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ldlm_poold S 0000000000000001 0 7105 2 0x00000000 2014-01-07 18:54:53 ffff881006cd1da0 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:53 ffff881006cd1d60 ffffffffa03847b1 0000000000000010 ffff881006cd1d70 2014-01-07 18:54:53 ffff881006cc8638 ffff881006cd1fd8 000000000000fb88 ffff881006cc8638 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:53 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:53 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:53 [] ldlm_pools_thread_main+0x1d0/0x2f0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ? ldlm_pools_thread_main+0x0/0x2f0 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_cfg_requeu S 0000000000000000 0 7106 2 0x00000000 2014-01-07 18:54:53 ffff881006cd5e40 0000000000000046 ffff881006cd5e40 ffffffff8150f36a 2014-01-07 18:54:53 0000000000000000 0000000000000282 0000000000000000 dead000000200200 2014-01-07 18:54:53 ffff881006cd3ab8 ffff881006cd5fd8 000000000000fb88 ffff881006cd3ab8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ? schedule_timeout+0x19a/0x2e0 2014-01-07 18:54:53 [] mgc_requeue_thread+0x4cc/0x700 [mgc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ? mgc_requeue_thread+0x0/0x700 [mgc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost00_000 S 0000000000000001 0 7109 2 0x00000000 2014-01-07 18:54:53 ffff881006eedda0 0000000000000046 0000000000000000 ffff881006eedd64 2014-01-07 18:54:53 ffff8807d16ff400 ffff88083fe81200 ffff880044656740 0000000000000400 2014-01-07 18:54:53 ffff881006cd3058 ffff881006eedfd8 000000000000fb88 ffff881006cd3058 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost00_001 S 0000000000000001 0 7110 2 0x00000000 2014-01-07 18:54:53 ffff881006eefda0 0000000000000046 ffff881006eefd68 ffff881006eefd64 2014-01-07 18:54:53 ffff880700000000 ffff88083fe81200 ffff880044656740 0000000000000400 2014-01-07 18:54:53 ffff881006cd25f8 ffff881006eeffd8 000000000000fb88 ffff881006cd25f8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost00_002 S 0000000000000002 0 7111 2 0x00000000 2014-01-07 18:54:53 ffff881006ef3da0 0000000000000046 0000000000000000 ffff881006ef3d64 2014-01-07 18:54:53 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:53 ffff881006ef1af8 ffff881006ef3fd8 000000000000fb88 ffff881006ef1af8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost01_000 S 0000000000000004 0 7112 2 0x00000000 2014-01-07 18:54:53 ffff881006ef9da0 0000000000000046 0000000000000000 ffff8807d060cb40 2014-01-07 18:54:53 ffff8807d16ff000 0000000000000000 000000000000002f ffff8807d1a36080 2014-01-07 18:54:53 ffff881006ef1098 ffff881006ef9fd8 000000000000fb88 ffff881006ef1098 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost01_001 S 0000000000000007 0 7113 2 0x00000000 2014-01-07 18:54:53 ffff881006efbda0 0000000000000046 ffff881006efbd68 ffff881006efbd64 2014-01-07 18:54:53 ffff8807d16ff000 ffff88083fe81e00 ffff880044696740 0000000000000400 2014-01-07 18:54:53 ffff881006ef0638 ffff881006efbfd8 000000000000fb88 ffff881006ef0638 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost01_002 S 0000000000000006 0 7114 2 0x00000000 2014-01-07 18:54:53 ffff881006effda0 0000000000000046 ffff881006effd68 ffff881006effd64 2014-01-07 18:54:53 ffff8807d16ff000 ffff88083fe81c00 ffff8800446f6740 0000000000000301 2014-01-07 18:54:53 ffff881006efdab8 ffff881006efffd8 000000000000fb88 ffff881006efdab8 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost02_000 S 0000000000000009 0 7115 2 0x00000000 2014-01-07 18:54:53 ffff881006f03da0 0000000000000046 ffff881006f03d68 ffff881006f03d64 2014-01-07 18:54:53 ffff881030b29c00 ffff88083fe82200 ffff880044636740 0000000000000406 2014-01-07 18:54:53 ffff881006efd058 ffff881006f03fd8 000000000000fb88 ffff881006efd058 2014-01-07 18:54:53 Call Trace: 2014-01-07 18:54:53 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:53 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:53 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:53 [] kthread+0x96/0xa0 2014-01-07 18:54:53 [] child_rip+0xa/0x20 2014-01-07 18:54:53 [] ? kthread+0x0/0xa0 2014-01-07 18:54:53 [] ? child_rip+0x0/0x20 2014-01-07 18:54:53 ll_ost02_001 S 000000000000000b 0 7116 2 0x00000000 2014-01-07 18:54:53 ffff881006f09da0 0000000000000046 ffff881006f09d68 ffff881006f09d64 2014-01-07 18:54:53 ffff881030b29c00 ffff88083fe82000 ffff88085c436740 0000000000000383 2014-01-07 18:54:53 ffff881006efc5f8 ffff881006f09fd8 000000000000fb88 ffff881006efc5f8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost02_002 S 000000000000000b 0 7117 2 0x00000000 2014-01-07 18:54:54 ffff881006f11da0 0000000000000046 ffff881006f11d68 ffff881006f11d64 2014-01-07 18:54:54 ffff881030b29c00 ffff88083fe82600 ffff88085c456740 0000000000000400 2014-01-07 18:54:54 ffff881006f0baf8 ffff881006f11fd8 000000000000fb88 ffff881006f0baf8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost03_000 S 000000000000000f 0 7118 2 0x00000000 2014-01-07 18:54:54 ffff881006f19da0 0000000000000046 0000000000000000 ffff881006f19d64 2014-01-07 18:54:54 ffff881000000000 ffff88083fe82800 ffff880044616740 0000000000000a06 2014-01-07 18:54:54 ffff881006f0b098 ffff881006f19fd8 000000000000fb88 ffff881006f0b098 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost03_001 S 000000000000000e 0 7119 2 0x00000000 2014-01-07 18:54:54 ffff881006f21da0 0000000000000046 0000000000000000 ffff881006f21d64 2014-01-07 18:54:54 ffff881031875000 ffff88083fe82e00 ffff88085c496740 0000000000000400 2014-01-07 18:54:54 ffff881006f0a638 ffff881006f21fd8 000000000000fb88 ffff881006f0a638 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost03_002 S 000000000000000d 0 7120 2 0x00000000 2014-01-07 18:54:54 ffff881006f29da0 0000000000000046 ffff881000000010 ffff881031e98740 2014-01-07 18:54:54 ffff881031875000 0000000000000000 0000000000000063 ffff8807d1a36080 2014-01-07 18:54:54 ffff881006f23ab8 ffff881006f29fd8 000000000000fb88 ffff881006f23ab8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:54 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 0000000000000000 0 7121 2 0x00000000 2014-01-07 18:54:54 ffff8810068cfda0 0000000000000046 0000000000000000 ffff88080e247480 2014-01-07 18:54:54 ffff8807c0202c00 0000000000000000 0000000000000011 ffff8807d1a35ec0 2014-01-07 18:54:54 ffff881006f23058 ffff8810068cffd8 000000000000fb88 ffff881006f23058 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 0000000000000002 0 7122 2 0x00000000 2014-01-07 18:54:54 ffff881006979da0 0000000000000046 0000000000000000 ffff881006979d64 2014-01-07 18:54:54 ffff880700000000 ffff88083fe81400 ffff88085c456740 00000000000009fd 2014-01-07 18:54:54 ffff881006f225f8 ffff881006979fd8 000000000000fb88 ffff881006f225f8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 0000000000000005 0 7123 2 0x00000000 2014-01-07 18:54:54 ffff88100697dda0 0000000000000046 0000000000000000 ffff8807d060c6c0 2014-01-07 18:54:54 ffff8807c0202800 0000000000000000 000000000000002f ffff8807d1a35ec0 2014-01-07 18:54:54 ffff88100697baf8 ffff88100697dfd8 000000000000fb88 ffff88100697baf8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 0000000000000005 0 7124 2 0x00000000 2014-01-07 18:54:54 ffff881006981da0 0000000000000046 0000000000000000 ffff8807d060c540 2014-01-07 18:54:54 ffff8807c0202800 0000000000000000 0000000000000010 ffff8807d1a35ec0 2014-01-07 18:54:54 ffff88100697b098 ffff881006981fd8 000000000000fb88 ffff88100697b098 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 000000000000000a 0 7125 2 0x00000000 2014-01-07 18:54:54 ffff881006983da0 0000000000000046 0000000000000000 ffff881006983d64 2014-01-07 18:54:54 ffff881031875400 ffff88083fe82400 ffff88085c476740 0000000000000400 2014-01-07 18:54:54 ffff88100697a638 ffff881006983fd8 000000000000fb88 ffff88100697a638 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 0000000000000009 0 7126 2 0x00000000 2014-01-07 18:54:54 ffff88100698dda0 0000000000000046 ffff881000000010 ffff881031ead200 2014-01-07 18:54:54 ffff881031875400 0000000000000000 0000000000000009 ffff8807d1a35ec0 2014-01-07 18:54:54 ffff881006987ab8 ffff88100698dfd8 000000000000fb88 ffff881006987ab8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 000000000000000d 0 7127 2 0x00000000 2014-01-07 18:54:54 ffff881006995da0 0000000000000046 0000000000000000 ffff88102f5618c0 2014-01-07 18:54:54 ffff881031875800 0000000000000000 000000000000001f ffff8807d1a35ec0 2014-01-07 18:54:54 ffff881006987058 ffff881006995fd8 000000000000fb88 ffff881006987058 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_create S 000000000000000c 0 7128 2 0x00000000 2014-01-07 18:54:54 ffff8810069b7da0 0000000000000046 ffff8810069b7d68 ffff8810069b7d64 2014-01-07 18:54:54 ffff881000000000 ffff88083fe82800 ffff88085c4b6740 0000000000000400 2014-01-07 18:54:54 ffff8810069865f8 ffff8810069b7fd8 000000000000fb88 ffff8810069865f8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_io00_0 S 0000000000000003 0 7129 2 0x00000000 2014-01-07 18:54:54 ffff881006223da0 0000000000000046 0000000000000000 ffff881006223d64 2014-01-07 18:54:54 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:54:54 ffff881006221af8 ffff881006223fd8 000000000000fb88 ffff881006221af8 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:54 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:54 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:54 [] kthread+0x96/0xa0 2014-01-07 18:54:54 [] child_rip+0xa/0x20 2014-01-07 18:54:54 [] ? kthread+0x0/0xa0 2014-01-07 18:54:54 [] ? child_rip+0x0/0x20 2014-01-07 18:54:54 ll_ost_io00_0 D 0000000000000000 0 7130 2 0x00000000 2014-01-07 18:54:54 ffff881006227360 0000000000000046 0000000000000000 ffff881006227324 2014-01-07 18:54:54 ffff881000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:54 ffff881006221098 ffff881006227fd8 000000000000fb88 ffff881006221098 2014-01-07 18:54:54 Call Trace: 2014-01-07 18:54:54 [] io_schedule+0x73/0xc0 2014-01-07 18:54:54 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:54 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:54 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:54 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:54 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:54:54 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:54:54 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:54:54 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:54:55 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:54:55 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:54:55 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:54:55 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:55 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:55 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:55 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:55 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:55 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:55 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io00_0 D 0000000000000002 0 7131 2 0x00000000 2014-01-07 18:54:55 ffff881006229690 0000000000000046 0000000000000000 ffff881006229654 2014-01-07 18:54:55 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:55 ffff881006220638 ffff881006229fd8 000000000000fb88 ffff881006220638 2014-01-07 18:54:55 Call Trace: 2014-01-07 18:54:55 [] io_schedule+0x73/0xc0 2014-01-07 18:54:55 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:55 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:55 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:55 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:55 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:55 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:55 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:55 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:55 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:55 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io01_0 D 0000000000000006 0 7132 2 0x00000000 2014-01-07 18:54:55 ffff88100622d690 0000000000000046 ffff88100622d6b0 ffffffffa04d9367 2014-01-07 18:54:55 ffff88100622d6b0 0000000000000001 0000000000000b24 ffff880700000000 2014-01-07 18:54:55 ffff88100622bab8 ffff88100622dfd8 000000000000fb88 ffff88100622bab8 2014-01-07 18:54:55 Call Trace: 2014-01-07 18:54:55 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:54:55 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:55 [] io_schedule+0x73/0xc0 2014-01-07 18:54:55 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:55 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:55 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:55 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:55 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:55 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:55 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:55 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:55 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:55 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io01_0 S 0000000000000004 0 7133 2 0x00000000 2014-01-07 18:54:55 ffff881006233930 0000000000000046 0000000000000000 ffff8810062338f4 2014-01-07 18:54:55 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:55 ffff88100622b058 ffff881006233fd8 000000000000fb88 ffff88100622b058 2014-01-07 18:54:55 Call Trace: 2014-01-07 18:54:55 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:55 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:55 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:54:55 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:54:55 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:55 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:54:55 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io01_0 S 0000000000000006 0 7134 2 0x00000000 2014-01-07 18:54:55 ffff881006235930 0000000000000046 0000000000000000 ffff8810062358f4 2014-01-07 18:54:55 00052ccb00000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:54:55 ffff88100622a5f8 ffff881006235fd8 000000000000fb88 ffff88100622a5f8 2014-01-07 18:54:55 Call Trace: 2014-01-07 18:54:55 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:55 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:55 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:54:55 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:54:55 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:55 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:54:55 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io02_0 D 0000000000000008 0 7135 2 0x00000000 2014-01-07 18:54:55 ffff881006239690 0000000000000046 0000000000000000 ffff881006239654 2014-01-07 18:54:55 ffff881000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:54:55 ffff881006237af8 ffff881006239fd8 000000000000fb88 ffff881006237af8 2014-01-07 18:54:55 Call Trace: 2014-01-07 18:54:55 [] io_schedule+0x73/0xc0 2014-01-07 18:54:55 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:55 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:55 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:55 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:55 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:55 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:55 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:55 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:55 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:55 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:55 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:55 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:55 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:55 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:55 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:55 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:55 [] kthread+0x96/0xa0 2014-01-07 18:54:55 [] child_rip+0xa/0x20 2014-01-07 18:54:55 [] ? kthread+0x0/0xa0 2014-01-07 18:54:55 [] ? child_rip+0x0/0x20 2014-01-07 18:54:55 ll_ost_io02_0 D 0000000000000008 0 7136 2 0x00000000 2014-01-07 18:54:55 ffff881006271690 0000000000000046 ffff881006271658 ffff881006271654 2014-01-07 18:54:55 ffff8810062716b0 ffff88083fe82000 ffff880044636740 0000000000000207 2014-01-07 18:54:56 ffff881006237098 ffffffffffffffff 00000000ffffffff 0000020000000800 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? pointer+0x8d/0x830 2014-01-07 18:54:56 [] ? pointer+0x8d/0x830 2014-01-07 18:54:56 [] ? put_dec+0x10c/0x110 2014-01-07 18:54:56 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:54:56 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:54:56 [] ? put_dec+0x10c/0x110 2014-01-07 18:54:56 [] ? put_dec+0x10c/0x110 2014-01-07 18:54:56 [] ? pointer+0x8d/0x830 2014-01-07 18:54:56 [] ? schedule_timeout+0x192/0x2e0 2014-01-07 18:54:56 [] ? put_dec+0x10c/0x110 2014-01-07 18:54:56 [] ? number+0x2ee/0x320 2014-01-07 18:54:56 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:54:56 [] ? put_dec+0x10c/0x110 2014-01-07 18:54:56 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:54:56 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:56 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:54:56 [] ? read_tsc+0x9/0x20 2014-01-07 18:54:56 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] ? ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:56 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:56 [] ? ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:56 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:56 [] ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_io02_0 S 0000000000000008 0 7137 2 0x00000000 2014-01-07 18:54:56 ffff881006273da0 0000000000000046 0000000000000000 ffff881032729a40 2014-01-07 18:54:56 ffff881031875c00 0000000000000000 000000000000000d ffff8807d1a359c0 2014-01-07 18:54:56 ffff881006236638 ffff881006273fd8 000000000000fb88 ffff881006236638 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:56 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_io03_0 D 000000000000000c 0 7138 2 0x00000000 2014-01-07 18:54:56 ffff88100627f690 0000000000000046 0000000000000000 ffff88100627f654 2014-01-07 18:54:56 ffff881000000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:54:56 ffff88100627dab8 ffff88100627ffd8 000000000000fb88 ffff88100627dab8 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] io_schedule+0x73/0xc0 2014-01-07 18:54:56 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:56 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:56 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:56 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:56 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:56 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:56 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:56 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:56 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:56 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:56 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:56 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:56 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:56 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:56 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:56 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_io03_0 S 000000000000000d 0 7139 2 0x00000000 2014-01-07 18:54:56 ffff8810062a3930 0000000000000046 0000000000000000 ffff8810062a38f4 2014-01-07 18:54:56 52ccbdf200000000 0000000000000286 ffff8810062a38d0 ffffffff810810cc 2014-01-07 18:54:56 ffff88100627d058 ffff8810062a3fd8 000000000000fb88 ffff88100627d058 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:54:56 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:56 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:56 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:54:56 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:54:56 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:56 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:54:56 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:56 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:56 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:56 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:56 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_io03_0 D 000000000000000c 0 7140 2 0x00000000 2014-01-07 18:54:56 ffff8810062ab690 0000000000000046 ffff8810062ab6b0 ffffffffa04d9367 2014-01-07 18:54:56 ffff8810062ab6b0 0000000000000001 0000000000000c3b ffff880e00000000 2014-01-07 18:54:56 ffff88100627c5f8 ffff8810062abfd8 000000000000fb88 ffff88100627c5f8 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:54:56 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:56 [] io_schedule+0x73/0xc0 2014-01-07 18:54:56 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:56 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:56 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:56 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:56 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:56 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:56 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:56 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:56 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:56 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:56 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:56 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:56 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:56 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:56 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:56 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:56 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:56 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_seq00_ S 0000000000000001 0 7141 2 0x00000000 2014-01-07 18:54:56 ffff881005d2dda0 0000000000000046 ffff881005d2dd20 ffffffff81055f96 2014-01-07 18:54:56 ffff881005d2dd30 ffff88083239f540 ffff881005d2dd30 ffffffff8105231d 2014-01-07 18:54:56 ffff881005c73af8 ffff881005d2dfd8 000000000000fb88 ffff881005c73af8 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:56 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:56 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:56 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:56 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:56 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_seq00_ S 0000000000000001 0 7142 2 0x00000000 2014-01-07 18:54:56 ffff881005d31da0 0000000000000046 ffff881005d31d20 ffffffff81055f96 2014-01-07 18:54:56 ffff881005d31d30 ffff88083239f540 ffff881005d31d30 ffffffff8105231d 2014-01-07 18:54:56 ffff881005c73098 ffff881005d31fd8 000000000000fb88 ffff881005c73098 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:56 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:56 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:56 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:56 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:56 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] kthread+0x96/0xa0 2014-01-07 18:54:56 [] child_rip+0xa/0x20 2014-01-07 18:54:56 [] ? kthread+0x0/0xa0 2014-01-07 18:54:56 [] ? child_rip+0x0/0x20 2014-01-07 18:54:56 ll_ost_seq01_ S 0000000000000004 0 7143 2 0x00000000 2014-01-07 18:54:56 ffff881005d33da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:56 ffff881005d33d30 ffff88083239f540 ffff881005d33d30 ffffffff8105231d 2014-01-07 18:54:56 ffff881005c72638 ffff881005d33fd8 000000000000fb88 ffff881005c72638 2014-01-07 18:54:56 Call Trace: 2014-01-07 18:54:56 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:56 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:56 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:56 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:56 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:56 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:56 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_ost_seq01_ S 0000000000000004 0 7144 2 0x00000000 2014-01-07 18:54:57 ffff881005d37da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:57 ffff881005d37d30 ffff88083239f540 ffff881005d37d30 ffffffff8105231d 2014-01-07 18:54:57 ffff881005d35ab8 ffff881005d37fd8 000000000000fb88 ffff881005d35ab8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:57 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:57 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:57 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_ost_seq02_ S 0000000000000008 0 7145 2 0x00000000 2014-01-07 18:54:57 ffff881005d3bda0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:57 ffff881005d3bd30 ffff88083239f540 ffff881005d3bd30 ffffffff8105231d 2014-01-07 18:54:57 ffff881005d35058 ffff881005d3bfd8 000000000000fb88 ffff881005d35058 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:57 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:57 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:57 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_ost_seq02_ S 0000000000000008 0 7146 2 0x00000000 2014-01-07 18:54:57 ffff881005d41da0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:57 ffff881005d41d30 ffff88083239f540 ffff881005d41d30 ffffffff8105231d 2014-01-07 18:54:57 ffff881005d345f8 ffff881005d41fd8 000000000000fb88 ffff881005d345f8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:57 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:57 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:57 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_ost_seq03_ S 000000000000000c 0 7147 2 0x00000000 2014-01-07 18:54:57 ffff881005d4bda0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:57 ffff881005d4bd30 ffff88083239f540 ffff881005d4bd30 ffffffff8105231d 2014-01-07 18:54:57 ffff881005d49af8 ffff881005d4bfd8 000000000000fb88 ffff881005d49af8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:57 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:57 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:57 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_ost_seq03_ S 000000000000000c 0 7148 2 0x00000000 2014-01-07 18:54:57 ffff881005d4fda0 0000000000000046 0000000000000000 ffffffff81055f96 2014-01-07 18:54:57 ffff881005d4fd30 ffff88083239f540 ffff881005d4fd30 ffffffff8105231d 2014-01-07 18:54:57 ffff881005d49098 ffff881005d4ffd8 000000000000fb88 ffff881005d49098 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? enqueue_task+0x66/0x80 2014-01-07 18:54:57 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:54:57 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? __wake_up+0x53/0x70 2014-01-07 18:54:57 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 ll_evictor S 0000000000000001 0 7149 2 0x00000000 2014-01-07 18:54:57 ffff881005d59e00 0000000000000046 ffff8800446367a8 ffff881005d480b8 2014-01-07 18:54:57 ffff881005d59da0 ffffffff81065463 ffff881005d59db0 0000000000000246 2014-01-07 18:54:57 ffff881005d48638 ffff881005d59fd8 000000000000fb88 ffff881005d48638 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] ? dequeue_entity+0x113/0x2e0 2014-01-07 18:54:57 [] ? ping_evictor_main+0x0/0x650 [ptlrpc] 2014-01-07 18:54:57 [] ping_evictor_main+0x4c3/0x650 [ptlrpc] 2014-01-07 18:54:57 [] ? thread_return+0x4e/0x76e 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? ping_evictor_main+0x0/0x650 [ptlrpc] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 lquota_wb_lus S 0000000000000000 0 7156 2 0x00000000 2014-01-07 18:54:57 ffff881005d61d40 0000000000000046 0000000000000000 ffff881005d61d04 2014-01-07 18:54:57 0000000000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:57 ffff881005d5a5f8 ffff881005d61fd8 000000000000fb88 ffff881005d5a5f8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:57 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:57 [] qsd_upd_thread+0x94d/0xd10 [lquota] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? qsd_upd_thread+0x0/0xd10 [lquota] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000000 0 7157 2 0x00000000 2014-01-07 18:54:57 ffff881005d65e20 0000000000000046 0000000000000030 0000000000000000 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d65e00 ffff8807d1ff6688 2014-01-07 18:54:57 ffff881005d5bab8 ffff881005d65fd8 000000000000fb88 ffff881005d5bab8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000001 0 7158 2 0x00000000 2014-01-07 18:54:57 ffff881005d5de20 0000000000000046 ffff88083febacc0 0000000000000282 2014-01-07 18:54:57 ffff88083fc00040 0000000300000000 ffff881005d5de00 ffff8807d1ff7608 2014-01-07 18:54:57 ffff88100728c638 ffff881005d5dfd8 000000000000fb88 ffff88100728c638 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000002 0 7159 2 0x00000000 2014-01-07 18:54:57 ffff881005d69e20 0000000000000046 0000000000000024 0000000000000000 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d69e00 ffff8807d1ff7308 2014-01-07 18:54:57 ffff88100728d098 ffff881005d69fd8 000000000000fb88 ffff88100728d098 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000003 0 7160 2 0x00000000 2014-01-07 18:54:57 ffff881005d6be20 0000000000000046 0000000000000007 ffff880e80fe8600 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d6be00 ffff8807d1ff6208 2014-01-07 18:54:57 ffff88100728daf8 ffff881005d6bfd8 000000000000fb88 ffff88100728daf8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000004 0 7161 2 0x00000000 2014-01-07 18:54:57 ffff881005d6de20 0000000000000046 0000000000000007 ffff880e8031b600 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d6de00 ffff8807d1ff7488 2014-01-07 18:54:57 ffff881030bb25f8 ffff881005d6dfd8 000000000000fb88 ffff881030bb25f8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000005 0 7162 2 0x00000000 2014-01-07 18:54:57 ffff881005d6fe20 0000000000000046 0000000000000001 ffff880e7ff4e800 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d6fe00 ffff8807d1ff7c08 2014-01-07 18:54:57 ffff8810302de638 ffff881005d6ffd8 000000000000fb88 ffff8810302de638 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000006 0 7163 2 0x00000000 2014-01-07 18:54:57 ffff881005d71e20 0000000000000046 ffff881005d71de8 ffff881005d71de4 2014-01-07 18:54:57 00000000fffffffe ffff88083fe81c00 ffff88085c436740 000000000000e87b 2014-01-07 18:54:57 ffff881030bf9af8 ffff881005d71fd8 000000000000fb88 ffff881030bf9af8 2014-01-07 18:54:57 Call Trace: 2014-01-07 18:54:57 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:57 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:57 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:57 [] kthread+0x96/0xa0 2014-01-07 18:54:57 [] child_rip+0xa/0x20 2014-01-07 18:54:57 [] ? kthread+0x0/0xa0 2014-01-07 18:54:57 [] ? child_rip+0x0/0x20 2014-01-07 18:54:57 tx_commit_cb/ S 0000000000000007 0 7164 2 0x00000000 2014-01-07 18:54:57 ffff881005d73e20 0000000000000046 0000000000000006 ffff880e8310f400 2014-01-07 18:54:57 00000000fffffffe 0000000300000000 ffff881005d73e00 ffff8807d1ff7a88 2014-01-07 18:54:58 ffff881030bb3058 ffff881005d73fd8 000000000000fb88 ffff881030bb3058 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 0000000000000008 0 7165 2 0x00000000 2014-01-07 18:54:58 ffff881005d75e20 0000000000000046 0000000000000026 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff881005d75e00 ffff8807d1ff7f08 2014-01-07 18:54:58 ffff88100725b098 ffff881005d75fd8 000000000000fb88 ffff88100725b098 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 0000000000000009 0 7166 2 0x00000000 2014-01-07 18:54:58 ffff881005d77e20 0000000000000046 0000000000000035 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff881005d77e00 ffff8807d1ff7cc8 2014-01-07 18:54:58 ffff8810309145f8 ffff881005d77fd8 000000000000fb88 ffff8810309145f8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000a 0 7167 2 0x00000000 2014-01-07 18:54:58 ffff881005d79e20 0000000000000046 0000000000000000 ffff88007ff97200 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff881005d79e00 ffff8807bf03c748 2014-01-07 18:54:58 ffff881030bb3ab8 ffff881005d79fd8 000000000000fb88 ffff881030bb3ab8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000b 0 7168 2 0x00000000 2014-01-07 18:54:58 ffff8807be5d5e20 0000000000000046 0000000000000026 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff8807be5d5e00 ffff8807d1ff6148 2014-01-07 18:54:58 ffff880828f4b058 ffff8807be5d5fd8 000000000000fb88 ffff880828f4b058 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000c 0 7169 2 0x00000000 2014-01-07 18:54:58 ffff8807be5d7e20 0000000000000046 0000000000000000 ffff8807be5d7de4 2014-01-07 18:54:58 0000000000000000 ffff88083fe82800 ffff88085c4d6740 000000000000ad64 2014-01-07 18:54:58 ffff8808335d05f8 ffff8807be5d7fd8 000000000000fb88 ffff8808335d05f8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000d 0 7170 2 0x00000000 2014-01-07 18:54:58 ffff8807be5d9e20 0000000000000046 0000000000000027 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff8807be5d9e00 ffff8807d1ff6988 2014-01-07 18:54:58 ffff880828f4bab8 ffff8807be5d9fd8 000000000000fb88 ffff880828f4bab8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000e 0 7171 2 0x00000000 2014-01-07 18:54:58 ffff8807be5dbe20 0000000000000046 0000000000000029 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff8807be5dbe00 ffff8807d1ff6088 2014-01-07 18:54:58 ffff8808325d3af8 ffff8807be5dbfd8 000000000000fb88 ffff8808325d3af8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 tx_commit_cb/ S 000000000000000f 0 7172 2 0x00000000 2014-01-07 18:54:58 ffff8807be5dde20 0000000000000046 000000000000002f 0000000000000000 2014-01-07 18:54:58 00000000fffffffe 0000000300000000 ffff8807be5dde00 ffff8807d1ff7848 2014-01-07 18:54:58 ffff8808335d1058 ffff8807be5ddfd8 000000000000fb88 ffff8808335d1058 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] taskq_thread+0x3c6/0x3f0 [spl] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? taskq_thread+0x0/0x3f0 [spl] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ldlm_cb01_002 S 0000000000000005 0 7217 2 0x00000000 2014-01-07 18:54:58 ffff8807d273fda0 0000000000000046 0000000000000000 ffff8807bf5b92c0 2014-01-07 18:54:58 ffff8807d16c0400 0000000000000000 0000000000000003 ffff8807d1a36380 2014-01-07 18:54:58 ffff88083239faf8 ffff8807d273ffd8 000000000000fb88 ffff88083239faf8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:58 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ldlm_bl_02 S 0000000000000000 0 7218 2 0x00000000 2014-01-07 18:54:58 ffff8807be687e40 0000000000000046 0000000000000000 0000000000000000 2014-01-07 18:54:58 0000000000000002 0000000000000000 ffff8807be687e10 ffff8800b9c69600 2014-01-07 18:54:58 ffff880831acdaf8 ffff8807be687fd8 000000000000fb88 ffff880831acdaf8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ldlm_bl_thread_main+0x345/0x3c0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ? ldlm_bl_thread_main+0x0/0x3c0 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ll_ost01_003 S 0000000000000006 0 7223 2 0x00000000 2014-01-07 18:54:58 ffff8807bdac7da0 0000000000000046 ffff8807bdac7d68 ffff8807bdac7d64 2014-01-07 18:54:58 ffff8807d16ff000 ffff88083fe81c00 ffff8800446f6740 0000000000000207 2014-01-07 18:54:58 ffff8808323d1098 ffff8807bdac7fd8 000000000000fb88 ffff8808323d1098 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ll_ost00_003 S 0000000000000002 0 7224 2 0x00000000 2014-01-07 18:54:58 ffff8807bd523da0 0000000000000046 0000000000000000 ffff8807bd523d64 2014-01-07 18:54:58 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:54:58 ffff8808319e8638 ffff8807bd523fd8 000000000000fb88 ffff8808319e8638 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ll_ost02_003 S 0000000000000008 0 7225 2 0x00000000 2014-01-07 18:54:58 ffff8807bc86bda0 0000000000000046 ffff881000000010 ffff881005f73240 2014-01-07 18:54:58 ffff881030b29c00 0000000000000000 000000000000003d ffff8807d1a36080 2014-01-07 18:54:58 ffff880833ec1ab8 ffff8807bc86bfd8 000000000000fb88 ffff880833ec1ab8 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:58 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:58 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ll_ost03_003 S 000000000000000d 0 7226 2 0x00000000 2014-01-07 18:54:58 ffff8807bbf2dda0 0000000000000046 0000000000000000 ffff8810058d8e40 2014-01-07 18:54:58 ffff881031875000 0000000000000000 0000000000000000 ffff8807d1a36080 2014-01-07 18:54:58 ffff880833fce638 ffff8807bbf2dfd8 000000000000fb88 ffff880833fce638 2014-01-07 18:54:58 Call Trace: 2014-01-07 18:54:58 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:58 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:58 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:58 [] kthread+0x96/0xa0 2014-01-07 18:54:58 [] child_rip+0xa/0x20 2014-01-07 18:54:58 [] ? kthread+0x0/0xa0 2014-01-07 18:54:58 [] ? child_rip+0x0/0x20 2014-01-07 18:54:58 ll_ost_create S 0000000000000009 0 7227 2 0x00000000 2014-01-07 18:54:58 ffff881005d97da0 0000000000000046 0000000000000000 ffff881032247e40 2014-01-07 18:54:58 ffff881031875400 0000000000000000 0000000000000008 ffff8807d1a35ec0 2014-01-07 18:54:58 ffff88100725baf8 ffff881005d97fd8 000000000000fb88 ffff88100725baf8 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_create S 0000000000000004 0 7228 2 0x00000000 2014-01-07 18:54:59 ffff881005453da0 0000000000000046 0000000000000000 ffff881005453d64 2014-01-07 18:54:59 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:59 ffff8810301f7098 ffff881005453fd8 000000000000fb88 ffff8810301f7098 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_create S 000000000000000f 0 7229 2 0x00000000 2014-01-07 18:54:59 ffff881005457da0 0000000000000046 0000000000000000 ffff8810058d8cc0 2014-01-07 18:54:59 ffff881031875800 0000000000000000 0000000000000000 ffff8807d1a35ec0 2014-01-07 18:54:59 ffff881005d5b058 ffff881005457fd8 000000000000fb88 ffff881005d5b058 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ? lprocfs_counter_add+0x1a8/0x1d6 [lvfs] 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_create S 0000000000000001 0 7230 2 0x00000000 2014-01-07 18:54:59 ffff88100546dda0 0000000000000046 0000000000000000 ffff8807ba945680 2014-01-07 18:54:59 ffff8807c0202c00 0000000000000000 0000000000000010 ffff8807d1a35ec0 2014-01-07 18:54:59 ffff881030a7d098 ffff88100546dfd8 000000000000fb88 ffff881030a7d098 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_create S 000000000000000b 0 7231 2 0x00000000 2014-01-07 18:54:59 ffff8810054cfda0 0000000000000046 0000000000000000 ffff881031dde3c0 2014-01-07 18:54:59 ffff881031875400 0000000000000000 0000000000000009 ffff8807d1a35ec0 2014-01-07 18:54:59 ffff8810054cdab8 ffff8810054cffd8 000000000000fb88 ffff8810054cdab8 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_io00_0 S 0000000000000000 0 7312 2 0x00000000 2014-01-07 18:54:59 ffff8810057d1da0 0000000000000046 0000000000000010 ffff8807ba9452c0 2014-01-07 18:54:59 ffff8807c0202400 0000000000000000 0000000000000055 ffff8807d1a359c0 2014-01-07 18:54:59 ffff881031a5d098 ffff8810057d1fd8 000000000000fb88 ffff881031a5d098 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:59 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_io00_0 S 0000000000000000 0 7313 2 0x00000000 2014-01-07 18:54:59 ffff8810057d7da0 0000000000000046 0000000000000000 ffff8810057d7d64 2014-01-07 18:54:59 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:54:59 ffff881030a07058 ffff8810057d7fd8 000000000000fb88 ffff881030a07058 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost01_004 S 0000000000000005 0 7314 2 0x00000000 2014-01-07 18:54:59 ffff881005759da0 0000000000000046 ffff880700000010 ffff8807baf62180 2014-01-07 18:54:59 ffff8807d16ff000 0000000000000000 0000000000000012 ffff8807d1a36080 2014-01-07 18:54:59 ffff881005771af8 ffff881005759fd8 000000000000fb88 ffff881005771af8 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:54:59 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_io01_0 S 0000000000000004 0 7315 2 0x00000000 2014-01-07 18:54:59 ffff8810057bb930 0000000000000046 0000000000000000 ffff8810057bb8f4 2014-01-07 18:54:59 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:59 ffff8810054cc5f8 ffff8810057bbfd8 000000000000fb88 ffff8810054cc5f8 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:54:59 [] ? process_timeout+0x0/0x10 2014-01-07 18:54:59 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:54:59 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:54:59 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:54:59 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:59 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:59 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:59 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:59 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:59 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_io02_0 D 0000000000000009 0 7316 2 0x00000000 2014-01-07 18:54:59 ffff881005725690 0000000000000046 ffff8810057256b0 ffffffffa04d9367 2014-01-07 18:54:59 ffff8810057256b0 0000000000000001 0000000000000ba2 ffff880e00000000 2014-01-07 18:54:59 ffff8810054cd058 ffff881005725fd8 000000000000fb88 ffff8810054cd058 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:54:59 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:54:59 [] io_schedule+0x73/0xc0 2014-01-07 18:54:59 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:54:59 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:54:59 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:54:59 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:54:59 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:54:59 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:54:59 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:54:59 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:54:59 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:54:59 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:54:59 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:59 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:54:59 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:54:59 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:54:59 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:54:59 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:54:59 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:54:59 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:54:59 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:54:59 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost02_004 S 0000000000000009 0 7317 2 0x00000000 2014-01-07 18:54:59 ffff8810057b7da0 0000000000000046 0000000000000000 ffff8810057b7d64 2014-01-07 18:54:59 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:54:59 ffff881031e905f8 ffff8810057b7fd8 000000000000fb88 ffff881031e905f8 2014-01-07 18:54:59 Call Trace: 2014-01-07 18:54:59 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:54:59 [] ? default_wake_function+0x0/0x20 2014-01-07 18:54:59 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:54:59 [] kthread+0x96/0xa0 2014-01-07 18:54:59 [] child_rip+0xa/0x20 2014-01-07 18:54:59 [] ? kthread+0x0/0xa0 2014-01-07 18:54:59 [] ? child_rip+0x0/0x20 2014-01-07 18:54:59 ll_ost_io01_0 S 0000000000000004 0 7318 2 0x00000000 2014-01-07 18:54:59 ffff88100509d930 0000000000000046 0000000000000000 ffff88100509d8f4 2014-01-07 18:54:59 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:54:59 ffff8810309c3058 ffff88100509dfd8 000000000000fb88 ffff8810309c3058 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:00 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:00 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:00 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:00 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:00 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:00 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:00 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:00 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:00 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:00 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io02_0 S 0000000000000008 0 7319 2 0x00000000 2014-01-07 18:55:00 ffff8810050a1da0 0000000000000046 ffff8810050a1d68 ffff8810050a1d64 2014-01-07 18:55:00 ffff881031875c00 ffff88083fe82000 ffff880044636740 0000000000000406 2014-01-07 18:55:00 ffff8810309c25f8 ffff8810050a1fd8 000000000000fb88 ffff8810309c25f8 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:00 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io00_0 S 0000000000000000 0 7320 2 0x00000000 2014-01-07 18:55:00 ffff8810050f7930 0000000000000046 0000000000000000 ffff8810050f78f4 2014-01-07 18:55:00 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:00 ffff881031a5c638 ffff8810050f7fd8 000000000000fb88 ffff881031a5c638 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:00 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:00 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:00 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:00 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:00 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:00 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:00 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:00 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:00 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:00 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io02_0 S 000000000000000a 0 7321 2 0x00000000 2014-01-07 18:55:00 ffff88100515fda0 0000000000000046 ffff881000000010 ffff8810058556c0 2014-01-07 18:55:00 ffff881031875c00 0000000000000000 000000000000002e ffff8807d1a359c0 2014-01-07 18:55:00 ffff88100515dab8 ffff88100515ffd8 000000000000fb88 ffff88100515dab8 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:00 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:00 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io01_0 D 0000000000000006 0 7322 2 0x00000000 2014-01-07 18:55:00 ffff8810051d5550 0000000000000046 0000000000000000 ffff8810051d5514 2014-01-07 18:55:00 0000000200000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:00 ffff88100515d058 ffff8810051d5fd8 000000000000fb88 ffff88100515d058 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] __mutex_lock_slowpath+0x13e/0x180 2014-01-07 18:55:00 [] mutex_lock+0x2b/0x50 2014-01-07 18:55:00 [] dmu_zfetch+0x40b/0xd70 [zfs] 2014-01-07 18:55:00 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:00 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:00 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:00 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:00 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:00 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:00 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:00 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:00 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:00 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:00 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:00 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io01_0 S 0000000000000006 0 7323 2 0x00000000 2014-01-07 18:55:00 ffff8810051d9930 0000000000000046 ffff8810051d98f8 ffff8810051d98f4 2014-01-07 18:55:00 00052ccb00000000 ffff88083fe81c00 ffff8800446b6740 0000000000000400 2014-01-07 18:55:00 ffff8810051d7af8 ffff8810051d9fd8 000000000000fb88 ffff8810051d7af8 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:00 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:00 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:00 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:00 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:00 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:00 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:00 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:00 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:00 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:00 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io03_0 D 000000000000000c 0 7324 2 0x00000000 2014-01-07 18:55:00 ffff881005263690 0000000000000046 0000000000000000 ffff881005263654 2014-01-07 18:55:00 ffff881000000000 ffff88083fe82800 ffff880044636740 00000000000001ff 2014-01-07 18:55:00 ffff88100515c5f8 ffff881005263fd8 000000000000fb88 ffff88100515c5f8 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] io_schedule+0x73/0xc0 2014-01-07 18:55:00 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:00 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:00 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:00 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:00 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:00 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:00 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:00 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:00 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:00 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:00 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:00 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:00 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:00 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:00 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:00 [] kthread+0x96/0xa0 2014-01-07 18:55:00 [] child_rip+0xa/0x20 2014-01-07 18:55:00 [] ? kthread+0x0/0xa0 2014-01-07 18:55:00 [] ? child_rip+0x0/0x20 2014-01-07 18:55:00 ll_ost_io02_0 D 0000000000000009 0 7325 2 0x00000000 2014-01-07 18:55:00 ffff8810052bd690 0000000000000046 0000000000000000 ffffffffa04d9367 2014-01-07 18:55:00 ffff8810052bd6b0 0000000000000002 0000000000000c66 ffff880e00000000 2014-01-07 18:55:00 ffff8810052bbab8 ffff8810052bdfd8 000000000000fb88 ffff8810052bbab8 2014-01-07 18:55:00 Call Trace: 2014-01-07 18:55:00 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:00 [] io_schedule+0x73/0xc0 2014-01-07 18:55:00 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:00 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:00 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:00 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:00 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:00 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:00 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:00 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:00 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:00 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:00 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:01 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:01 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io00_0 S 0000000000000000 0 7326 2 0x00000000 2014-01-07 18:55:01 ffff881005375930 0000000000000046 0000000000000000 ffff8810053758f4 2014-01-07 18:55:01 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:01 ffff8810052bb058 ffff881005375fd8 000000000000fb88 ffff8810052bb058 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:01 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:01 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:01 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:01 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io00_0 D 0000000000000002 0 7327 2 0x00000000 2014-01-07 18:55:01 ffff8810053fb690 0000000000000046 0000000000000000 ffff8810053fb654 2014-01-07 18:55:01 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:01 ffff8810052ba5f8 ffff8810053fbfd8 000000000000fb88 ffff8810052ba5f8 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] io_schedule+0x73/0xc0 2014-01-07 18:55:01 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:01 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:01 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:01 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:01 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:01 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:01 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:01 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:01 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:01 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io01_0 S 0000000000000005 0 7328 2 0x00000000 2014-01-07 18:55:01 ffff881004c3d930 0000000000000046 0000000000000000 ffff881004c3d8f4 2014-01-07 18:55:01 0000000000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:01 ffff8810051d7098 ffff881004c3dfd8 000000000000fb88 ffff8810051d7098 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:01 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:01 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:01 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:01 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io02_0 D 0000000000000009 0 7329 2 0x00000000 2014-01-07 18:55:01 ffff881004cbd690 0000000000000046 0000000000000000 ffff881004cbd654 2014-01-07 18:55:01 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:55:01 ffff8810051d6638 ffff881004cbdfd8 000000000000fb88 ffff8810051d6638 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] io_schedule+0x73/0xc0 2014-01-07 18:55:01 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:01 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:01 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:01 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:01 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:01 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:01 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:01 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:01 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:01 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io02_0 D 0000000000000009 0 7330 2 0x00000000 2014-01-07 18:55:01 ffff881004cf3690 0000000000000046 0000000000000000 ffff881004cf3654 2014-01-07 18:55:01 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:55:01 ffff881004cf1af8 ffff881004cf3fd8 000000000000fb88 ffff881004cf1af8 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] io_schedule+0x73/0xc0 2014-01-07 18:55:01 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:01 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:01 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:01 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:01 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:01 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:01 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:01 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:01 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:01 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:01 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:01 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:01 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:01 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:01 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io02_0 S 000000000000000b 0 7331 2 0x00000000 2014-01-07 18:55:01 ffff881004d11da0 0000000000000046 0000000000000000 ffff881005287800 2014-01-07 18:55:01 ffff881031875c00 0000000000000000 0000000000000039 ffff8807d1a359c0 2014-01-07 18:55:01 ffff881004cffab8 ffff881004d11fd8 000000000000fb88 ffff881004cffab8 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:01 [] kthread+0x96/0xa0 2014-01-07 18:55:01 [] child_rip+0xa/0x20 2014-01-07 18:55:01 [] ? kthread+0x0/0xa0 2014-01-07 18:55:01 [] ? child_rip+0x0/0x20 2014-01-07 18:55:01 ll_ost_io03_0 S 000000000000000e 0 7332 2 0x00000000 2014-01-07 18:55:01 ffff881004d53930 0000000000000046 ffff881004d538f8 ffff881004d538f4 2014-01-07 18:55:01 52ccbdfb00000000 ffff88083fe82c00 ffff880044636740 0000000000000406 2014-01-07 18:55:01 ffff881004cff058 ffff881004d53fd8 000000000000fb88 ffff881004cff058 2014-01-07 18:55:01 Call Trace: 2014-01-07 18:55:01 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:01 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:01 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:01 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:01 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:01 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:01 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:01 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io03_0 D 000000000000000c 0 7333 2 0x00000000 2014-01-07 18:55:02 ffff881004d5d690 0000000000000046 ffff881004d5d6b0 ffffffffa04d9367 2014-01-07 18:55:02 ffff881004d5d6b0 0000000000000001 0000000000000b5a ffff880700000000 2014-01-07 18:55:02 ffff881004cf1098 ffff881004d5dfd8 000000000000fb88 ffff881004cf1098 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:02 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:02 [] io_schedule+0x73/0xc0 2014-01-07 18:55:02 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:02 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:02 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:02 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:02 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:02 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:02 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:02 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:02 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:02 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:02 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io03_0 S 000000000000000d 0 7334 2 0x00000000 2014-01-07 18:55:02 ffff881004dc3930 0000000000000046 ffff881004dc38f8 ffff881004dc38f4 2014-01-07 18:55:02 52ccbe0000000000 ffff88083fe82a00 ffff880044616740 0000000000000207 2014-01-07 18:55:02 ffff881004cf0638 ffff881004dc3fd8 000000000000fb88 ffff881004cf0638 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:02 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:02 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:02 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:02 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:02 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:02 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io01_0 S 0000000000000005 0 7335 2 0x00000000 2014-01-07 18:55:02 ffff881004e19930 0000000000000046 0000000000000000 ffff881004e198f4 2014-01-07 18:55:02 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:02 ffff881004e17af8 ffff881004e19fd8 000000000000fb88 ffff881004e17af8 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:02 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:02 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:02 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:02 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:02 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:02 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io01_0 D 0000000000000004 0 7336 2 0x00000000 2014-01-07 18:55:02 ffff881004e1d690 0000000000000046 ffff881004e1d6b0 ffffffffa04d9367 2014-01-07 18:55:02 ffff881004e1d6b0 0000000000000001 0000000000000c04 ffff880e00000000 2014-01-07 18:55:02 ffff881004e17098 ffff881004e1dfd8 000000000000fb88 ffff881004e17098 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:02 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:02 [] io_schedule+0x73/0xc0 2014-01-07 18:55:02 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:02 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:02 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:02 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:02 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:02 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:02 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:02 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:02 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:02 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:02 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:02 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io00_0 S 0000000000000002 0 7337 2 0x00000000 2014-01-07 18:55:02 ffff881004e25930 0000000000000046 0000000000000000 ffff881004e258f4 2014-01-07 18:55:02 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:02 ffff881004e23ab8 ffff881004e25fd8 000000000000fb88 ffff881004e23ab8 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:02 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:02 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:02 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:02 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:02 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:02 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io00_0 S 0000000000000000 0 7338 2 0x00000000 2014-01-07 18:55:02 ffff881004e49930 0000000000000046 0000000000000000 ffff881004e498f4 2014-01-07 18:55:02 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:02 ffff881004cfe5f8 ffff881004e49fd8 000000000000fb88 ffff881004cfe5f8 2014-01-07 18:55:02 Call Trace: 2014-01-07 18:55:02 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:02 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:02 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:02 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:02 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:02 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:02 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:02 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:02 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:02 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:02 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:02 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:02 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:02 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:02 [] kthread+0x96/0xa0 2014-01-07 18:55:02 [] child_rip+0xa/0x20 2014-01-07 18:55:02 [] ? kthread+0x0/0xa0 2014-01-07 18:55:02 [] ? child_rip+0x0/0x20 2014-01-07 18:55:02 ll_ost_io00_0 S 0000000000000002 0 7339 2 0x00000000 2014-01-07 18:55:02 ffff881004e4d930 0000000000000046 0000000000000000 ffff881004e4d8f4 2014-01-07 18:55:02 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:03 ffff881004e4baf8 ffff881004e4dfd8 000000000000fb88 ffff881004e4baf8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:03 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:03 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:03 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:03 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:03 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:03 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost_io01_0 S 0000000000000005 0 7340 2 0x00000000 2014-01-07 18:55:03 ffff8807ac271930 0000000000000046 0000000000000000 ffff8807ac2718f4 2014-01-07 18:55:03 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:03 ffff8808271b3af8 ffff8807ac271fd8 000000000000fb88 ffff8808271b3af8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:03 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:03 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:03 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:03 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:03 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:03 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost_io01_0 D 0000000000000006 0 7341 2 0x00000000 2014-01-07 18:55:03 ffff8807aaf7b690 0000000000000046 0000000000000000 ffff8807aaf7b654 2014-01-07 18:55:03 ffff880700000000 ffff88083fe81c00 ffff880044636740 0000000000000181 2014-01-07 18:55:03 ffff8808343bb058 ffff8807aaf7bfd8 000000000000fb88 ffff8808343bb058 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] io_schedule+0x73/0xc0 2014-01-07 18:55:03 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:03 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:03 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:03 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:03 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:03 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:03 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:03 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:03 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:03 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:03 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:03 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:03 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:03 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost00_004 S 0000000000000001 0 7342 2 0x00000000 2014-01-07 18:55:03 ffff8807a9cfdda0 0000000000000046 ffff8807a9cfdd68 ffff8807a9cfdd64 2014-01-07 18:55:03 ffff8807d16ff400 ffff88083fe81200 ffff880044656740 0000000000000400 2014-01-07 18:55:03 ffff880832af9af8 ffff8807a9cfdfd8 000000000000fb88 ffff880832af9af8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost03_004 S 000000000000000e 0 7343 2 0x00000000 2014-01-07 18:55:03 ffff8807a914fda0 0000000000000046 0000000000000000 ffff8807a914fd64 2014-01-07 18:55:03 ffff881031875000 ffff88083fe82c00 ffff880044636740 0000000000000207 2014-01-07 18:55:03 ffff880832af9098 ffff8807a914ffd8 000000000000fb88 ffff880832af9098 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost_io03_0 S 000000000000000e 0 7344 2 0x00000000 2014-01-07 18:55:03 ffff8807a8d91930 0000000000000046 ffff8807a8d918f8 ffff8807a8d918f4 2014-01-07 18:55:03 52ccbde800000000 ffff88083fe82c00 ffff880044616740 0000000000000207 2014-01-07 18:55:03 ffff88082721bab8 ffff8807a8d91fd8 000000000000fb88 ffff88082721bab8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:03 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:03 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:03 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:03 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:03 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:03 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost_io00_0 D 0000000000000000 0 7345 2 0x00000000 2014-01-07 18:55:03 ffff8807a8f83690 0000000000000046 ffff8807a8f836b0 ffffffffa04d9367 2014-01-07 18:55:03 ffff8807a8f836b0 0000000000000001 0000000000000af1 ffff880e00000000 2014-01-07 18:55:03 ffff880827311ab8 ffff8807a8f83fd8 000000000000fb88 ffff880827311ab8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:03 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:03 [] io_schedule+0x73/0xc0 2014-01-07 18:55:03 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:03 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:03 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:03 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:03 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:03 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:03 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:03 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:03 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:03 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:03 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:03 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:03 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:03 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:03 [] kthread+0x96/0xa0 2014-01-07 18:55:03 [] child_rip+0xa/0x20 2014-01-07 18:55:03 [] ? kthread+0x0/0xa0 2014-01-07 18:55:03 [] ? child_rip+0x0/0x20 2014-01-07 18:55:03 ll_ost_io03_0 S 000000000000000d 0 7346 2 0x00000000 2014-01-07 18:55:03 ffff8807a8a27930 0000000000000046 ffff8807a8a278f8 ffff8807a8a278f4 2014-01-07 18:55:03 00052ccb00000000 ffff88083fe82a00 ffff880044616740 0000000000000207 2014-01-07 18:55:03 ffff8808273105f8 ffff8807a8a27fd8 000000000000fb88 ffff8808273105f8 2014-01-07 18:55:03 Call Trace: 2014-01-07 18:55:03 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:03 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:03 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:03 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:03 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:03 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:03 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:03 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:03 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:03 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:03 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io00_0 S 0000000000000002 0 7347 2 0x00000000 2014-01-07 18:55:04 ffff8807a7e27da0 0000000000000046 ffff880700000010 ffff8807b0ced540 2014-01-07 18:55:04 ffff8807c0202400 0000000000000000 0000000000000048 ffff8807d1a359c0 2014-01-07 18:55:04 ffff880831907058 ffff8807a7e27fd8 000000000000fb88 ffff880831907058 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:04 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:04 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:04 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io00_0 S 0000000000000002 0 7348 2 0x00000000 2014-01-07 18:55:04 ffff8807a79c9930 0000000000000046 0000000000000000 ffff8807a79c98f4 2014-01-07 18:55:04 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:04 ffff8808343a3058 ffff8807a79c9fd8 000000000000fb88 ffff8808343a3058 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:04 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:04 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:04 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:04 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:04 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:04 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:04 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io02_0 D 0000000000000008 0 7349 2 0x00000000 2014-01-07 18:55:04 ffff8807a7a83690 0000000000000046 ffff8807a7a836b0 ffffffffa04d9367 2014-01-07 18:55:04 ffff8807a7a836b0 0000000000000001 0000000000000e15 ffff880e00000000 2014-01-07 18:55:04 ffff8808343a25f8 ffff8807a7a83fd8 000000000000fb88 ffff8808343a25f8 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:04 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:04 [] io_schedule+0x73/0xc0 2014-01-07 18:55:04 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:04 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:04 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:04 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:04 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:04 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:04 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:04 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:04 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:04 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:04 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io03_0 S 000000000000000c 0 7350 2 0x00000000 2014-01-07 18:55:04 ffff8807a6eeb930 0000000000000046 0000000000000000 ffff8807a6eeb8f4 2014-01-07 18:55:04 00052ccb00000000 0000000000000286 ffff8807a6eeb8d0 ffffffff810810cc 2014-01-07 18:55:04 ffff88082836bab8 ffff8807a6eebfd8 000000000000fb88 ffff88082836bab8 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:04 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:04 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:04 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:04 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:04 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:04 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:04 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:04 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io00_0 D 0000000000000003 0 7351 2 0x00000000 2014-01-07 18:55:04 ffff8807a6eef360 0000000000000046 0000000000000000 ffff8807a6eef324 2014-01-07 18:55:04 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:04 ffff8808335f3098 ffff8807a6eeffd8 000000000000fb88 ffff8808335f3098 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] io_schedule+0x73/0xc0 2014-01-07 18:55:04 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:04 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:04 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:04 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:04 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:04 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:04 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:04 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:04 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:04 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:04 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:04 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:04 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:04 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:04 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:04 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:04 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:04 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:04 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io01_0 D 0000000000000005 0 7352 2 0x00000000 2014-01-07 18:55:04 ffff8807a635b690 0000000000000046 ffff8807a635b6b0 ffffffffa04d9367 2014-01-07 18:55:04 ffff8807a635b6b0 0000000000000001 0000000000000c6a ffff880e00000000 2014-01-07 18:55:04 ffff8808335f3af8 ffff8807a635bfd8 000000000000fb88 ffff8808335f3af8 2014-01-07 18:55:04 Call Trace: 2014-01-07 18:55:04 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:04 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:04 [] io_schedule+0x73/0xc0 2014-01-07 18:55:04 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:04 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:04 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:04 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:04 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:04 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:04 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:04 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:04 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:04 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:04 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:04 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:04 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:04 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:04 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:04 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:04 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:04 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:04 [] kthread+0x96/0xa0 2014-01-07 18:55:04 [] child_rip+0xa/0x20 2014-01-07 18:55:04 [] ? kthread+0x0/0xa0 2014-01-07 18:55:04 [] ? child_rip+0x0/0x20 2014-01-07 18:55:04 ll_ost_io01_0 D 0000000000000006 0 7353 2 0x00000000 2014-01-07 18:55:04 ffff8807a5f93690 0000000000000046 0000000000000000 ffff8807a5f93654 2014-01-07 18:55:04 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff88083127e5f8 ffff8807a5f93fd8 000000000000fb88 ffff88083127e5f8 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] io_schedule+0x73/0xc0 2014-01-07 18:55:05 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:05 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:05 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:05 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:05 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:05 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:05 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:05 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:05 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:05 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:05 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:05 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:05 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:05 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io00_0 S 0000000000000003 0 7354 2 0x00000000 2014-01-07 18:55:05 ffff8807a5f95da0 0000000000000046 0000000000000000 ffff8807a5f95d64 2014-01-07 18:55:05 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff88083127fab8 ffff8807a5f95fd8 000000000000fb88 ffff88083127fab8 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:05 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io01_0 S 0000000000000007 0 7355 2 0x00000000 2014-01-07 18:55:05 ffff8807a5bed930 0000000000000046 0000000000000000 ffff8807a5bed8f4 2014-01-07 18:55:05 00052ccb00000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff8808329d85f8 ffff8807a5bedfd8 000000000000fb88 ffff8808329d85f8 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:05 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:05 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:05 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:05 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:05 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:05 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:05 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:05 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:05 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:05 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io00_0 S 0000000000000002 0 7356 2 0x00000000 2014-01-07 18:55:05 ffff8807a4c47da0 0000000000000046 0000000000000000 ffff8807a4c47d64 2014-01-07 18:55:05 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff88082721b058 ffff8807a4c47fd8 000000000000fb88 ffff88082721b058 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:05 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io00_0 S 0000000000000000 0 7357 2 0x00000000 2014-01-07 18:55:05 ffff8807a489fda0 0000000000000046 0000000000000000 ffff8807a489fd64 2014-01-07 18:55:05 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff880831a64638 ffff8807a489ffd8 000000000000fb88 ffff880831a64638 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:05 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io01_0 S 0000000000000005 0 7358 2 0x00000000 2014-01-07 18:55:05 ffff8807a48b9930 0000000000000046 0000000000000000 ffff8807a48b98f4 2014-01-07 18:55:05 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff88083355a638 ffff8807a48b9fd8 000000000000fb88 ffff88083355a638 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:05 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:05 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:05 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:05 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:05 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:05 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:05 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:05 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:05 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:05 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io00_0 D 0000000000000002 0 7359 2 0x00000000 2014-01-07 18:55:05 ffff8807a40f5690 0000000000000046 0000000000000000 ffff8807a40f5654 2014-01-07 18:55:05 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:05 ffff8808335e7ab8 ffff8807a40f5fd8 000000000000fb88 ffff8808335e7ab8 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] io_schedule+0x73/0xc0 2014-01-07 18:55:05 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:05 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:05 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:05 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:05 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:05 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:05 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:05 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:05 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:05 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:05 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:05 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:05 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:05 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:05 [] kthread+0x96/0xa0 2014-01-07 18:55:05 [] child_rip+0xa/0x20 2014-01-07 18:55:05 [] ? kthread+0x0/0xa0 2014-01-07 18:55:05 [] ? child_rip+0x0/0x20 2014-01-07 18:55:05 ll_ost_io03_0 D 000000000000000f 0 7360 2 0x00000000 2014-01-07 18:55:05 ffff8807a3589360 0000000000000046 ffff8807a3589328 ffff8807a3589324 2014-01-07 18:55:05 ffff880700000000 ffff88083fe82e00 ffff880044616740 000000000000020f 2014-01-07 18:55:05 ffff880828e69ab8 ffff8807a3589fd8 000000000000fb88 ffff880828e69ab8 2014-01-07 18:55:05 Call Trace: 2014-01-07 18:55:05 [] io_schedule+0x73/0xc0 2014-01-07 18:55:05 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:05 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:05 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:05 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:05 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:05 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:05 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:05 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:05 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:05 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:05 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:05 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:05 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:05 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:05 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:05 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:05 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:05 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:05 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:05 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:05 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 S 000000000000000c 0 7361 2 0x00000000 2014-01-07 18:55:06 ffff8807a31c7930 0000000000000046 0000000000000000 ffff8807a31c78f4 2014-01-07 18:55:06 00052ccb00000000 ffff88083fe82800 ffff88085c476740 000000000000015a 2014-01-07 18:55:06 ffff880828e69058 ffff8807a31c7fd8 000000000000fb88 ffff880828e69058 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:06 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:06 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:06 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:06 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:06 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:06 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 S 000000000000000d 0 7362 2 0x00000000 2014-01-07 18:55:06 ffff8807a31cb930 0000000000000046 ffff8807a31cb8f8 ffff8807a31cb8f4 2014-01-07 18:55:06 52ccbe0000000000 0000000000000286 ffff8807a31cb8d0 ffffffff810810cc 2014-01-07 18:55:06 ffff880833fc45f8 ffff8807a31cbfd8 000000000000fb88 ffff880833fc45f8 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:06 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:06 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:06 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:06 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:06 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:06 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:06 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 S 000000000000000e 0 7363 2 0x00000000 2014-01-07 18:55:06 ffff8807a29f9930 0000000000000046 ffff8807a29f98f8 ffff8807a29f98f4 2014-01-07 18:55:06 52ccbdf600000000 ffff88083fe82c00 ffff880044616740 000000000000020f 2014-01-07 18:55:06 ffff88083264a5f8 ffff8807a29f9fd8 000000000000fb88 ffff88083264a5f8 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:06 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:06 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:06 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:06 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:06 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:06 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:06 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 S 000000000000000d 0 7364 2 0x00000000 2014-01-07 18:55:06 ffff8807a2619930 0000000000000046 ffff8807a26198f8 ffff8807a26198f4 2014-01-07 18:55:06 00052ccb00000000 0000000000000286 ffff8807a26198d0 ffffffff810810cc 2014-01-07 18:55:06 ffff880832f6d098 ffff8807a2619fd8 000000000000fb88 ffff880832f6d098 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:06 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:06 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:06 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:06 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:06 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:06 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:06 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io00_0 D 0000000000000000 0 7365 2 0x00000000 2014-01-07 18:55:06 ffff8807a1a67690 0000000000000046 0000000000000000 ffff8807a1a67654 2014-01-07 18:55:06 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:06 ffff880831a65098 ffff8807a1a67fd8 000000000000fb88 ffff880831a65098 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] io_schedule+0x73/0xc0 2014-01-07 18:55:06 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:06 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:06 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:06 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:06 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:06 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:06 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:06 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:06 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:06 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:06 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:06 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 S 000000000000000d 0 7366 2 0x00000000 2014-01-07 18:55:06 ffff8807a1a6d930 0000000000000046 ffff8807a1a6d8f8 ffff8807a1a6d8f4 2014-01-07 18:55:06 52ccbe0200000000 ffff88083fe82a00 ffff880044616740 000000000000040f 2014-01-07 18:55:06 ffff880833609098 ffff8807a1a6dfd8 000000000000fb88 ffff880833609098 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:06 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:06 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:06 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:06 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:06 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:06 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:06 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:06 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:06 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:06 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:06 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:06 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:06 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:06 [] kthread+0x96/0xa0 2014-01-07 18:55:06 [] child_rip+0xa/0x20 2014-01-07 18:55:06 [] ? kthread+0x0/0xa0 2014-01-07 18:55:06 [] ? child_rip+0x0/0x20 2014-01-07 18:55:06 ll_ost_io03_0 D 000000000000000c 0 7367 2 0x00000000 2014-01-07 18:55:06 ffff8807a1687690 0000000000000046 ffff8807a16876b0 ffffffffa04d9367 2014-01-07 18:55:06 ffff8807a16876b0 0000000000000001 0000000000000bc6 ffff880e00000000 2014-01-07 18:55:06 ffff88083128c638 ffff8807a1687fd8 000000000000fb88 ffff88083128c638 2014-01-07 18:55:06 Call Trace: 2014-01-07 18:55:06 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:06 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:06 [] io_schedule+0x73/0xc0 2014-01-07 18:55:06 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:06 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:06 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:06 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:07 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:07 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:07 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:07 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:07 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:07 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:07 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:07 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:07 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 D 0000000000000009 0 7368 2 0x00000000 2014-01-07 18:55:07 ffff8807a168b690 0000000000000046 ffff8807a168b658 ffff8807a168b654 2014-01-07 18:55:07 ffff8807a168b6b0 ffff88083fe82200 ffff88085c416740 00000000000002af 2014-01-07 18:55:07 ffff88083128d098 ffff8807a168bfd8 000000000000fb88 ffff88083128d098 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] io_schedule+0x73/0xc0 2014-01-07 18:55:07 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:07 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:07 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:07 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:07 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:07 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:07 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:07 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:07 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:07 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:07 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:07 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:07 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 S 0000000000000008 0 7369 2 0x00000000 2014-01-07 18:55:07 ffff8807a0b43da0 0000000000000046 0000000000000000 ffff881004ecd5c0 2014-01-07 18:55:07 ffff881031875c00 0000000000000000 0000000000000011 ffff8807d1a359c0 2014-01-07 18:55:07 ffff880833cdbab8 ffff8807a0b43fd8 000000000000fb88 ffff880833cdbab8 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:07 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io00_0 D 0000000000000003 0 7370 2 0x00000000 2014-01-07 18:55:07 ffff88079fb7b690 0000000000000046 0000000000000000 ffff88079fb7b654 2014-01-07 18:55:07 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:07 ffff880834305098 ffff88079fb7bfd8 000000000000fb88 ffff880834305098 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] io_schedule+0x73/0xc0 2014-01-07 18:55:07 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:07 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:07 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:07 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:07 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:07 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:07 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:07 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:07 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:07 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:07 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:07 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:07 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 D 0000000000000008 0 7371 2 0x00000000 2014-01-07 18:55:07 ffff88079f7b5360 0000000000000046 0000000000000004 0000000000000001 2014-01-07 18:55:07 ffff88079f7b52e0 ffffffff8150fa5e ffff880e8f7357c0 ffff880e8f735ad8 2014-01-07 18:55:07 ffff8808343bbab8 ffff88079f7b5fd8 000000000000fb88 ffff8808343bbab8 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] ? mutex_lock+0x1e/0x50 2014-01-07 18:55:07 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:07 [] io_schedule+0x73/0xc0 2014-01-07 18:55:07 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:07 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:07 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:07 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:07 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:07 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:07 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:07 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:07 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:07 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:07 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:07 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:07 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:07 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:07 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:07 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:07 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:07 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:07 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:07 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:07 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:07 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:07 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 S 0000000000000009 0 7372 2 0x00000000 2014-01-07 18:55:07 ffff88079e81bda0 0000000000000046 0000000000000000 ffff88100529d980 2014-01-07 18:55:07 ffff881031875c00 0000000000000000 000000000000003e ffff8807d1a359c0 2014-01-07 18:55:07 ffff88083435b098 ffff88079e81bfd8 000000000000fb88 ffff88083435b098 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:07 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 S 0000000000000009 0 7373 2 0x00000000 2014-01-07 18:55:07 ffff88079e81dda0 0000000000000046 0000000000000000 ffff88079e81dd64 2014-01-07 18:55:07 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:55:07 ffff88083435baf8 ffff88079e81dfd8 000000000000fb88 ffff88083435baf8 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:07 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:07 [] kthread+0x96/0xa0 2014-01-07 18:55:07 [] child_rip+0xa/0x20 2014-01-07 18:55:07 [] ? kthread+0x0/0xa0 2014-01-07 18:55:07 [] ? child_rip+0x0/0x20 2014-01-07 18:55:07 ll_ost_io02_0 S 0000000000000008 0 7374 2 0x00000000 2014-01-07 18:55:07 ffff88100474fda0 0000000000000046 ffff88100474fd68 ffff88100474fd64 2014-01-07 18:55:07 ffff881000000000 ffff88083fe82000 ffff880044616740 000000000000020f 2014-01-07 18:55:07 ffff881030a825f8 ffff88100474ffd8 000000000000fb88 ffff881030a825f8 2014-01-07 18:55:07 Call Trace: 2014-01-07 18:55:07 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:07 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:07 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io01_0 S 0000000000000004 0 7375 2 0x00000000 2014-01-07 18:55:08 ffff8810047e1930 0000000000000046 0000000000000000 ffff8810047e18f4 2014-01-07 18:55:08 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:08 ffff881030a83058 ffff8810047e1fd8 000000000000fb88 ffff881030a83058 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:08 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:08 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:08 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:08 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:08 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:08 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io01_0 D 0000000000000006 0 7376 2 0x00000000 2014-01-07 18:55:08 ffff881004001690 0000000000000046 0000000000000000 ffff881004001654 2014-01-07 18:55:08 ffff881000000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:08 ffff881032632638 ffff881004001fd8 000000000000fb88 ffff881032632638 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] io_schedule+0x73/0xc0 2014-01-07 18:55:08 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:08 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:08 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:08 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:08 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:08 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:08 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:08 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:08 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io00_0 D 0000000000000002 0 7377 2 0x00000000 2014-01-07 18:55:08 ffff8810040a5690 0000000000000046 0000000000000000 ffff8810040a5654 2014-01-07 18:55:08 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:08 ffff8810040a3ab8 ffff8810040a5fd8 000000000000fb88 ffff8810040a3ab8 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] io_schedule+0x73/0xc0 2014-01-07 18:55:08 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:08 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:08 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:08 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:08 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:08 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:08 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:08 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:08 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io03_0 S 000000000000000c 0 7378 2 0x00000000 2014-01-07 18:55:08 ffff8810040cb930 0000000000000046 0000000000000000 ffff8810040cb8f4 2014-01-07 18:55:08 00052ccb00000000 ffff88083fe82800 ffff88085c476740 000000000000015a 2014-01-07 18:55:08 ffff8810040c9af8 ffff8810040cbfd8 000000000000fb88 ffff8810040c9af8 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:08 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:08 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:08 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:08 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:08 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:08 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io03_0 D 000000000000000c 0 7379 2 0x00000000 2014-01-07 18:55:08 ffff881004143690 0000000000000046 ffff881004143658 ffff881004143654 2014-01-07 18:55:08 ffff881000000000 ffff88083fe82800 ffff880044616740 000000000000020f 2014-01-07 18:55:08 ffff8810040c9098 ffff881004143fd8 000000000000fb88 ffff8810040c9098 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] io_schedule+0x73/0xc0 2014-01-07 18:55:08 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:08 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:08 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:08 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:08 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:08 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:08 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:08 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:08 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:08 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:08 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io03_0 S 000000000000000d 0 7380 2 0x00000000 2014-01-07 18:55:08 ffff881004177930 0000000000000046 ffff8810041778f8 ffff8810041778f4 2014-01-07 18:55:08 52ccbe0a00000000 ffff88083fe82a00 ffff880044616740 000000000000020f 2014-01-07 18:55:08 ffff8810040c8638 ffff881004177fd8 000000000000fb88 ffff8810040c8638 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:08 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:08 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:08 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:08 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:08 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:08 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:08 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:08 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:08 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:08 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:08 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:08 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:08 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:08 [] kthread+0x96/0xa0 2014-01-07 18:55:08 [] child_rip+0xa/0x20 2014-01-07 18:55:08 [] ? kthread+0x0/0xa0 2014-01-07 18:55:08 [] ? child_rip+0x0/0x20 2014-01-07 18:55:08 ll_ost_io03_0 S 000000000000000d 0 7381 2 0x00000000 2014-01-07 18:55:08 ffff8810041b5930 0000000000000046 ffff8810041b58f8 ffff8810041b58f4 2014-01-07 18:55:08 00052ccb00000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:08 ffff8810041b3ab8 ffff8810041b5fd8 000000000000fb88 ffff8810041b3ab8 2014-01-07 18:55:08 Call Trace: 2014-01-07 18:55:08 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:08 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:08 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:08 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:08 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:08 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:08 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io00_0 S 0000000000000002 0 7382 2 0x00000000 2014-01-07 18:55:09 ffff8810041d1930 0000000000000046 0000000000000000 ffff8810041d18f4 2014-01-07 18:55:09 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:09 ffff881030379af8 ffff8810041d1fd8 000000000000fb88 ffff881030379af8 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:09 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:09 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:09 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:09 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io00_0 S 0000000000000002 0 7383 2 0x00000000 2014-01-07 18:55:09 ffff8810041d3da0 0000000000000046 0000000000000000 ffff8810041d3d64 2014-01-07 18:55:09 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:09 ffff8810321cd058 ffff8810041d3fd8 000000000000fb88 ffff8810321cd058 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io03_0 S 000000000000000d 0 7384 2 0x00000000 2014-01-07 18:55:09 ffff8810041f3930 0000000000000046 ffff8810041f38f8 ffff8810041f38f4 2014-01-07 18:55:09 00052ccb00000000 ffff88083fe82a00 ffff880044616740 000000000000020f 2014-01-07 18:55:09 ffff881005771098 ffff8810041f3fd8 000000000000fb88 ffff881005771098 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:09 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:09 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:09 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:09 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io00_0 S 0000000000000002 0 7385 2 0x00000000 2014-01-07 18:55:09 ffff88100422f930 0000000000000046 0000000000000000 ffff88100422f8f4 2014-01-07 18:55:09 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:09 ffff881005770638 ffff88100422ffd8 000000000000fb88 ffff881005770638 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:09 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:09 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:09 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:09 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io01_0 S 0000000000000004 0 7386 2 0x00000000 2014-01-07 18:55:09 ffff88100428f930 0000000000000046 0000000000000000 ffff88100428f8f4 2014-01-07 18:55:09 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:09 ffff881030cb7098 ffff88100428ffd8 000000000000fb88 ffff881030cb7098 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:09 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:09 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:09 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:09 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:09 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:09 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io01_0 D 0000000000000006 0 7387 2 0x00000000 2014-01-07 18:55:09 ffff8810042b5690 0000000000000046 0000000000000000 ffff8810042b5654 2014-01-07 18:55:09 ffff881000000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:09 ffff8810042b3af8 ffff8810042b5fd8 000000000000fb88 ffff8810042b3af8 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] io_schedule+0x73/0xc0 2014-01-07 18:55:09 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:09 [] ? zio_wait+0x123/0x1b0 [zfs] 2014-01-07 18:55:09 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:09 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:09 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:09 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:09 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:09 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:09 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:09 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:09 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:09 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:09 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:09 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:09 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:09 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:09 [] kthread+0x96/0xa0 2014-01-07 18:55:09 [] child_rip+0xa/0x20 2014-01-07 18:55:09 [] ? kthread+0x0/0xa0 2014-01-07 18:55:09 [] ? child_rip+0x0/0x20 2014-01-07 18:55:09 ll_ost_io02_0 D 0000000000000009 0 7388 2 0x00000000 2014-01-07 18:55:09 ffff8810042b7690 0000000000000046 0000000000000000 ffff8810042b7654 2014-01-07 18:55:09 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:09 ffff8810042b3098 ffff8810042b7fd8 000000000000fb88 ffff8810042b3098 2014-01-07 18:55:09 Call Trace: 2014-01-07 18:55:09 [] io_schedule+0x73/0xc0 2014-01-07 18:55:09 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:09 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:09 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:09 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:09 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:09 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:09 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:09 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:09 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:09 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:09 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:09 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:09 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:09 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:09 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io02_0 S 000000000000000a 0 7389 2 0x00000000 2014-01-07 18:55:10 ffff8810042f3930 0000000000000046 0000000000000001 ffff88102f4e2b40 2014-01-07 18:55:10 00052ccb77421d95 0000000000000286 ffff8810042f38d0 ffffffff8100bb8e 2014-01-07 18:55:10 ffff8810040a3058 ffff8810042f3fd8 000000000000fb88 ffff8810040a3058 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] ? apic_timer_interrupt+0xe/0x20 2014-01-07 18:55:10 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:10 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:10 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:10 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:10 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:10 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io02_0 S 0000000000000008 0 7390 2 0x00000000 2014-01-07 18:55:10 ffff88100438bda0 0000000000000046 0000000000000000 ffff881004ecdbc0 2014-01-07 18:55:10 ffff881031875c00 0000000000000000 000000000000002e ffff8807d1a359c0 2014-01-07 18:55:10 ffff8810040a25f8 ffff88100438bfd8 000000000000fb88 ffff8810040a25f8 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io02_0 S 000000000000000a 0 7391 2 0x00000000 2014-01-07 18:55:10 ffff8810043a5930 0000000000000046 ffff8810043a58f8 ffff8810043a58f4 2014-01-07 18:55:10 00052ccb00000000 ffff88083fe82400 ffff880044616740 000000000000020f 2014-01-07 18:55:10 ffff8810043a3ab8 ffff8810043a5fd8 000000000000fb88 ffff8810043a3ab8 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:10 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:10 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:10 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:10 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:10 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io03_0 S 000000000000000c 0 7392 2 0x00000000 2014-01-07 18:55:10 ffff881003c29930 0000000000000046 ffff881003c298f8 ffff881003c298f4 2014-01-07 18:55:10 52ccbdf900000000 ffff88083fe82800 ffff880044616740 000000000000020f 2014-01-07 18:55:10 ffff8810043a3058 ffff881003c29fd8 000000000000fb88 ffff8810043a3058 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:10 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:10 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:10 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:10 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:10 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io03_0 S 000000000000000c 0 7393 2 0x00000000 2014-01-07 18:55:10 ffff881003ca1930 0000000000000046 ffff881003ca18f8 ffff881003ca18f4 2014-01-07 18:55:10 00052ccb00000000 ffff88083fe82800 ffff880044616740 000000000000020f 2014-01-07 18:55:10 ffff8810043a25f8 ffff881003ca1fd8 000000000000fb88 ffff8810043a25f8 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:10 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:10 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:10 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:10 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:10 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io02_0 S 0000000000000008 0 7394 2 0x00000000 2014-01-07 18:55:10 ffff881003ca9da0 0000000000000046 0000000000000000 ffff8810047f3380 2014-01-07 18:55:10 ffff881031875c00 0000000000000000 000000000000004c ffff8807d1a359c0 2014-01-07 18:55:10 ffff881003ca3af8 ffff881003ca9fd8 000000000000fb88 ffff881003ca3af8 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io03_0 S 000000000000000d 0 7395 2 0x00000000 2014-01-07 18:55:10 ffff881003cd5930 0000000000000046 ffff881003cd58f8 ffff881003cd58f4 2014-01-07 18:55:10 52ccbde300000000 ffff88083fe82a00 ffff880044616740 000000000000020f 2014-01-07 18:55:10 ffff881003ca3098 ffff881003cd5fd8 000000000000fb88 ffff881003ca3098 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:10 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:10 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:10 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:10 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:10 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:10 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:10 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:10 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:10 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:10 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:10 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:10 [] kthread+0x96/0xa0 2014-01-07 18:55:10 [] child_rip+0xa/0x20 2014-01-07 18:55:10 [] ? kthread+0x0/0xa0 2014-01-07 18:55:10 [] ? child_rip+0x0/0x20 2014-01-07 18:55:10 ll_ost_io03_0 D 000000000000000c 0 7396 2 0x00000000 2014-01-07 18:55:10 ffff881003cdf690 0000000000000046 ffff881003cdf658 ffff881003cdf654 2014-01-07 18:55:10 ffff881000000000 ffff88083fe82800 ffff880044616740 000000000000040f 2014-01-07 18:55:10 ffff881004e16638 ffff881003cdffd8 000000000000fb88 ffff881004e16638 2014-01-07 18:55:10 Call Trace: 2014-01-07 18:55:10 [] io_schedule+0x73/0xc0 2014-01-07 18:55:10 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:10 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:10 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:10 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:10 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:10 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:10 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:10 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:10 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:10 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:10 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:10 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:10 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:10 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io02_0 S 0000000000000009 0 7397 2 0x00000000 2014-01-07 18:55:11 ffff881003dc1da0 0000000000000046 0000000000000000 ffff8810047f3b00 2014-01-07 18:55:11 ffff881031875c00 0000000000000000 0000000000000039 ffff8807d1a359c0 2014-01-07 18:55:11 ffff881004e4b098 ffff881003dc1fd8 000000000000fb88 ffff881004e4b098 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io00_0 S 0000000000000000 0 7398 2 0x00000000 2014-01-07 18:55:11 ffff881003e35930 0000000000000046 0000000000000000 ffff881003e358f4 2014-01-07 18:55:11 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:11 ffff881004e4a638 ffff881003e35fd8 000000000000fb88 ffff881004e4a638 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:11 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:11 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:11 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:11 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io01_0 D 0000000000000005 0 7399 2 0x00000000 2014-01-07 18:55:11 ffff881003809690 0000000000000046 0000000000000000 ffff881003809654 2014-01-07 18:55:11 ffff881000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:11 ffff881003807ab8 ffff881003809fd8 000000000000fb88 ffff881003807ab8 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] io_schedule+0x73/0xc0 2014-01-07 18:55:11 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:11 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:11 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:11 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:11 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:11 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:11 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:11 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:11 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:11 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:11 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io01_0 D 0000000000000005 0 7400 2 0x00000000 2014-01-07 18:55:11 ffff88100380f690 0000000000000046 ffff88100380f6b0 ffffffffa04d9367 2014-01-07 18:55:11 ffff88100380f6b0 0000000000000001 0000000000000b2e ffff880e00000000 2014-01-07 18:55:11 ffff88100380daf8 ffff88100380ffd8 000000000000fb88 ffff88100380daf8 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:11 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:11 [] io_schedule+0x73/0xc0 2014-01-07 18:55:11 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:11 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:11 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:11 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:11 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:11 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:11 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:11 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:11 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:11 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:11 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:11 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io01_0 S 0000000000000004 0 7401 2 0x00000000 2014-01-07 18:55:11 ffff881003833930 0000000000000046 0000000000000000 ffff8810038338f4 2014-01-07 18:55:11 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:11 ffff88100380d098 ffff881003833fd8 000000000000fb88 ffff88100380d098 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:11 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:11 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:11 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:11 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io01_0 S 0000000000000006 0 7402 2 0x00000000 2014-01-07 18:55:11 ffff881003835da0 0000000000000046 0000000000000000 ffff881003835d64 2014-01-07 18:55:11 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:11 ffff88100380c638 ffff881003835fd8 000000000000fb88 ffff88100380c638 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io00_0 S 0000000000000002 0 7403 2 0x00000000 2014-01-07 18:55:11 ffff881003931da0 0000000000000046 0000000000000000 ffff881003931d64 2014-01-07 18:55:11 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:11 ffff88100392fab8 ffff881003931fd8 000000000000fb88 ffff88100392fab8 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:11 [] ? child_rip+0x0/0x20 2014-01-07 18:55:11 ll_ost_io00_0 S 0000000000000003 0 7404 2 0x00000000 2014-01-07 18:55:11 ffff881003935930 0000000000000046 0000000000000000 ffff8810039358f4 2014-01-07 18:55:11 00052ccb00000000 ffff88083fe81600 ffff88085c496740 00000000000001f0 2014-01-07 18:55:11 ffff88100392f058 ffff881003935fd8 000000000000fb88 ffff88100392f058 2014-01-07 18:55:11 Call Trace: 2014-01-07 18:55:11 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:11 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:11 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:11 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:11 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:11 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:11 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:11 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:11 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:11 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:11 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:11 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:11 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:11 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:11 [] kthread+0x96/0xa0 2014-01-07 18:55:11 [] child_rip+0xa/0x20 2014-01-07 18:55:11 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io00_0 S 0000000000000003 0 7405 2 0x00000000 2014-01-07 18:55:12 ffff881003937930 0000000000000046 0000000000000000 ffff8810039378f4 2014-01-07 18:55:12 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:12 ffff88100392e5f8 ffff881003937fd8 000000000000fb88 ffff88100392e5f8 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:12 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:12 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:12 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:12 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:12 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:12 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:12 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io00_0 S 0000000000000000 0 7406 2 0x00000000 2014-01-07 18:55:12 ffff88100393d930 0000000000000046 0000000000000000 ffff88100393d8f4 2014-01-07 18:55:12 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:12 ffff88100393baf8 ffff88100393dfd8 000000000000fb88 ffff88100393baf8 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:12 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:12 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:12 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:12 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:12 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:12 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:12 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io00_0 S 0000000000000002 0 7407 2 0x00000000 2014-01-07 18:55:12 ffff88100395fda0 0000000000000046 0000000000000000 ffff88100395fd64 2014-01-07 18:55:12 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:12 ffff88100393b098 ffff88100395ffd8 000000000000fb88 ffff88100393b098 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io02_0 S 000000000000000a 0 7408 2 0x00000000 2014-01-07 18:55:12 ffff881003baf930 0000000000000046 0000000000000000 ffff881003baf8f4 2014-01-07 18:55:12 00052ccb00000000 ffff88083fe82400 ffff88085c476740 0000000000000180 2014-01-07 18:55:12 ffff88100393a638 ffff881003baffd8 000000000000fb88 ffff88100393a638 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:12 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:12 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:12 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:12 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:12 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:12 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:12 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io02_0 S 0000000000000009 0 7409 2 0x00000000 2014-01-07 18:55:12 ffff881003bc7da0 0000000000000046 0000000000000000 ffff881003d168c0 2014-01-07 18:55:12 ffff881031875c00 0000000000000000 0000000000000034 ffff8807d1a359c0 2014-01-07 18:55:12 ffff8810041b3058 ffff881003bc7fd8 000000000000fb88 ffff8810041b3058 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io02_0 S 0000000000000008 0 7410 2 0x00000000 2014-01-07 18:55:12 ffff881003bcf930 0000000000000046 ffff881003bcf8f8 ffff881003bcf8f4 2014-01-07 18:55:12 00052ccb00000000 ffff88083fe82000 ffff880044616740 0000000000000617 2014-01-07 18:55:12 ffff881004e23058 ffff881003bcffd8 000000000000fb88 ffff881004e23058 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:12 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:12 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:12 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:12 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:12 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:12 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:12 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io03_0 D 000000000000000c 0 7411 2 0x00000000 2014-01-07 18:55:12 ffff881003473690 0000000000000046 ffff881003473658 ffff881003473654 2014-01-07 18:55:12 ffff881000000000 ffff88083fe82800 ffff880044616740 0000000000000817 2014-01-07 18:55:12 ffff881004e225f8 ffff881003473fd8 000000000000fb88 ffff881004e225f8 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] io_schedule+0x73/0xc0 2014-01-07 18:55:12 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:12 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:12 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:12 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:12 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:12 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:12 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:12 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:12 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:12 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:12 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:12 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:12 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:12 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:12 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:12 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:12 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:12 [] kthread+0x96/0xa0 2014-01-07 18:55:12 [] child_rip+0xa/0x20 2014-01-07 18:55:12 [] ? kthread+0x0/0xa0 2014-01-07 18:55:12 [] ? child_rip+0x0/0x20 2014-01-07 18:55:12 ll_ost_io03_0 D 000000000000000d 0 7412 2 0x00000000 2014-01-07 18:55:12 ffff88100347b690 0000000000000046 ffff88100347b658 ffff88100347b654 2014-01-07 18:55:12 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:12 ffff881003479ab8 ffff88100347bfd8 000000000000fb88 ffff881003479ab8 2014-01-07 18:55:12 Call Trace: 2014-01-07 18:55:12 [] io_schedule+0x73/0xc0 2014-01-07 18:55:12 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:12 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:12 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:12 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:12 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:12 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:12 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:12 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:12 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:12 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:12 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:12 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:12 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:12 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:13 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:13 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:13 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:13 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:13 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:13 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io03_0 S 000000000000000c 0 7413 2 0x00000000 2014-01-07 18:55:13 ffff8810034c1930 0000000000000046 ffff8810034c18f8 ffff8810034c18f4 2014-01-07 18:55:13 52ccbdfa00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:13 ffff881003479058 ffff8810034c1fd8 000000000000fb88 ffff881003479058 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:13 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:13 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:13 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:13 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:13 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:13 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:13 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:13 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:13 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:13 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 000000000000000a 0 7414 2 0x00000000 2014-01-07 18:55:13 ffff8810034fbda0 0000000000000046 0000000000000000 ffff881005142ac0 2014-01-07 18:55:13 ffff881000000000 ffff88083fe82200 ffff88085c416740 0000000000000400 2014-01-07 18:55:13 ffff881003ca2638 ffff8810034fbfd8 000000000000fb88 ffff881003ca2638 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 0000000000000008 0 7415 2 0x00000000 2014-01-07 18:55:13 ffff881003551da0 0000000000000046 ffff881003551d68 ffff881003551d64 2014-01-07 18:55:13 ffff881031875c00 ffff88083fe82000 ffff88085c476740 0000000000000201 2014-01-07 18:55:13 ffff88100354baf8 ffff881003551fd8 000000000000fb88 ffff88100354baf8 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 0000000000000008 0 7416 2 0x00000000 2014-01-07 18:55:13 ffff881003559da0 0000000000000046 ffff881003559d68 ffff881003559d64 2014-01-07 18:55:13 ffff881031875c00 ffff88083fe82000 ffff88085c476740 0000000000000400 2014-01-07 18:55:13 ffff88100354b098 ffff881003559fd8 000000000000fb88 ffff88100354b098 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io01_0 S 0000000000000006 0 7417 2 0x00000000 2014-01-07 18:55:13 ffff88100355b930 0000000000000046 0000000000000000 ffff88100355b8f4 2014-01-07 18:55:13 00052ccb00000000 0000000000000286 ffff88100355b8d0 ffffffff810810cc 2014-01-07 18:55:13 ffff88100354a638 ffff88100355bfd8 000000000000fb88 ffff88100354a638 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:13 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:13 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:13 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:13 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:13 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:13 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:13 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:13 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:13 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:13 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:13 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 000000000000000a 0 7418 2 0x00000000 2014-01-07 18:55:13 ffff881003583da0 0000000000000046 ffff881000000010 ffff881005dd1680 2014-01-07 18:55:13 ffff881031875c00 0000000000000000 000000000000004d ffff8807d1a359c0 2014-01-07 18:55:13 ffff881003581ab8 ffff881003583fd8 000000000000fb88 ffff881003581ab8 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:13 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 000000000000000b 0 7419 2 0x00000000 2014-01-07 18:55:13 ffff881003587da0 0000000000000046 0000000000000000 ffff8810051cb600 2014-01-07 18:55:13 ffff881031875c00 0000000000000000 0000000000000035 ffff8807d1a359c0 2014-01-07 18:55:13 ffff881003581058 ffff881003587fd8 000000000000fb88 ffff881003581058 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io02_0 S 0000000000000009 0 7420 2 0x00000000 2014-01-07 18:55:13 ffff8810035b3da0 0000000000000046 ffff8810035b3d68 ffff8810035b3d64 2014-01-07 18:55:13 ffff881000000000 ffff88083fe82200 ffff880044616740 000000000000020f 2014-01-07 18:55:13 ffff8810035b1af8 ffff8810035b3fd8 000000000000fb88 ffff8810035b1af8 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io03_0 S 000000000000000d 0 7421 2 0x00000000 2014-01-07 18:55:13 ffff8810035b7930 0000000000000046 0000000000000000 ffff8810035b78f4 2014-01-07 18:55:13 52ccbe0a00000000 ffff88083fe82a00 ffff88085c476740 000000000000015a 2014-01-07 18:55:13 ffff8810035b1098 ffff8810035b7fd8 000000000000fb88 ffff8810035b1098 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:13 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:13 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:13 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:13 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:13 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:13 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:13 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:13 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:13 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:13 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io01_0 S 0000000000000006 0 7422 2 0x00000000 2014-01-07 18:55:13 ffff8810035ffda0 0000000000000046 0000000000000000 ffff8810035ffd64 2014-01-07 18:55:13 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:13 ffff8810035b0638 ffff8810035fffd8 000000000000fb88 ffff8810035b0638 2014-01-07 18:55:13 Call Trace: 2014-01-07 18:55:13 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:13 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:13 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:13 [] kthread+0x96/0xa0 2014-01-07 18:55:13 [] child_rip+0xa/0x20 2014-01-07 18:55:13 [] ? kthread+0x0/0xa0 2014-01-07 18:55:13 [] ? child_rip+0x0/0x20 2014-01-07 18:55:13 ll_ost_io03_0 D 000000000000000f 0 7423 2 0x00000000 2014-01-07 18:55:13 ffff881003603690 0000000000000046 ffff8810036036b0 ffffffffa04d9367 2014-01-07 18:55:13 ffff8810036036b0 0000000000000001 0000000000000e9b ffff880e00000000 2014-01-07 18:55:14 ffff881003601ab8 ffff881003603fd8 000000000000fb88 ffff881003601ab8 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:14 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:14 [] io_schedule+0x73/0xc0 2014-01-07 18:55:14 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:14 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:14 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:14 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:14 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:14 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:14 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:14 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:14 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:14 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:14 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io02_0 D 000000000000000a 0 7424 2 0x00000000 2014-01-07 18:55:14 ffff88100360d360 0000000000000046 ffff88100360d328 ffff88100360d324 2014-01-07 18:55:14 ffff881000000000 ffff88083fe82400 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff881003601058 ffff88100360dfd8 000000000000fb88 ffff881003601058 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] io_schedule+0x73/0xc0 2014-01-07 18:55:14 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:14 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:14 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:14 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:14 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:14 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:14 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:14 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:14 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:14 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:14 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:14 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:14 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:14 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:14 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:14 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:14 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:14 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:14 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:14 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io03_0 S 000000000000000c 0 7425 2 0x00000000 2014-01-07 18:55:14 ffff88100360f930 0000000000000046 ffff88100360f8f8 ffff88100360f8f4 2014-01-07 18:55:14 52ccbe0f00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff8810036005f8 ffff88100360ffd8 000000000000fb88 ffff8810036005f8 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:14 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:14 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:14 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:14 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:14 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:14 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io03_0 S 000000000000000e 0 7426 2 0x00000000 2014-01-07 18:55:14 ffff881003669930 0000000000000046 ffff8810036698f8 ffff8810036698f4 2014-01-07 18:55:14 52ccbe1000000000 ffff88083fe82c00 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff8810035805f8 ffff881003669fd8 000000000000fb88 ffff8810035805f8 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:14 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:14 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:14 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:14 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:14 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:14 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io03_0 S 000000000000000c 0 7427 2 0x00000000 2014-01-07 18:55:14 ffff881003683930 0000000000000046 ffff8810036838f8 ffff8810036838f4 2014-01-07 18:55:14 52ccbe0100000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff881003681af8 ffff881003683fd8 000000000000fb88 ffff881003681af8 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:14 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:14 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:14 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:14 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:14 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:14 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io03_0 S 000000000000000d 0 7428 2 0x00000000 2014-01-07 18:55:14 ffff881003687930 0000000000000046 ffff8810036878f8 ffff8810036878f4 2014-01-07 18:55:14 52ccbe0f00000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff881003681098 ffff881003687fd8 000000000000fb88 ffff881003681098 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:14 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:14 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:14 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:14 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:14 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:14 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:14 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:14 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:14 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:14 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:14 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:14 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:14 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:14 [] kthread+0x96/0xa0 2014-01-07 18:55:14 [] child_rip+0xa/0x20 2014-01-07 18:55:14 [] ? kthread+0x0/0xa0 2014-01-07 18:55:14 [] ? child_rip+0x0/0x20 2014-01-07 18:55:14 ll_ost_io03_0 S 000000000000000c 0 7429 2 0x00000000 2014-01-07 18:55:14 ffff8810036b1930 0000000000000046 ffff8810036b18f8 ffff8810036b18f4 2014-01-07 18:55:14 52ccbe0a00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:14 ffff881003680638 ffff8810036b1fd8 000000000000fb88 ffff881003680638 2014-01-07 18:55:14 Call Trace: 2014-01-07 18:55:14 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:14 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:14 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:14 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:15 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:15 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:15 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:15 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 S 0000000000000008 0 7430 2 0x00000000 2014-01-07 18:55:15 ffff8810036b9da0 0000000000000046 0000000000000000 ffff881003549600 2014-01-07 18:55:15 ffff881031875c00 0000000000000000 000000000000001f ffff8807d1a359c0 2014-01-07 18:55:15 ffff8810036b3ab8 ffff8810036b9fd8 000000000000fb88 ffff8810036b3ab8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 S 0000000000000008 0 7431 2 0x00000000 2014-01-07 18:55:15 ffff8810036e1da0 0000000000000046 0000000000000000 ffff8810036e1d64 2014-01-07 18:55:15 ffff881000000000 ffff88083fe82000 ffff88085c476740 000000000000015a 2014-01-07 18:55:15 ffff8810034785f8 ffff8810036e1fd8 000000000000fb88 ffff8810034785f8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 S 0000000000000008 0 7432 2 0x00000000 2014-01-07 18:55:15 ffff8810036e9da0 0000000000000046 0000000000000000 ffff8810051cb300 2014-01-07 18:55:15 ffff881031875c00 0000000000000000 0000000000000049 ffff8807d1a359c0 2014-01-07 18:55:15 ffff8810036e3af8 ffff8810036e9fd8 000000000000fb88 ffff8810036e3af8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 S 0000000000000009 0 7433 2 0x00000000 2014-01-07 18:55:15 ffff881003711da0 0000000000000046 0000000000000000 ffff881003711d64 2014-01-07 18:55:15 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:55:15 ffff8810036b3058 ffff881003711fd8 000000000000fb88 ffff8810036b3058 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io01_0 S 0000000000000004 0 7434 2 0x00000000 2014-01-07 18:55:15 ffff881003713930 0000000000000046 0000000000000000 ffff8810037138f4 2014-01-07 18:55:15 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:15 ffff8810036b25f8 ffff881003713fd8 000000000000fb88 ffff8810036b25f8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:15 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:15 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:15 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:15 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:15 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:15 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:15 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io01_0 S 0000000000000004 0 7435 2 0x00000000 2014-01-07 18:55:15 ffff88100371b930 0000000000000046 0000000000000000 ffff88100371b8f4 2014-01-07 18:55:15 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:15 ffff881003719ab8 ffff88100371bfd8 000000000000fb88 ffff881003719ab8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:15 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:15 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:15 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:15 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:15 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:15 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:15 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:15 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:15 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 D 0000000000000009 0 7436 2 0x00000000 2014-01-07 18:55:15 ffff88100371f690 0000000000000046 ffff88100371f6b0 ffffffffa04d9367 2014-01-07 18:55:15 ffff88100371f6b0 0000000000000001 0000000000000afd ffff880e00000000 2014-01-07 18:55:15 ffff881003719058 ffff88100371ffd8 000000000000fb88 ffff881003719058 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:15 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:15 [] io_schedule+0x73/0xc0 2014-01-07 18:55:15 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:15 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:15 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:15 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:15 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:15 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:15 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:15 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:15 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:15 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:15 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:15 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:15 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:15 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io01_0 D 0000000000000005 0 7437 2 0x00000000 2014-01-07 18:55:15 ffff881003733690 0000000000000046 ffff881003733658 ffff881003733654 2014-01-07 18:55:15 ffff8810037336b0 ffff88083fe81a00 ffff880044616740 0000000000000207 2014-01-07 18:55:15 ffff8810037185f8 ffff881003733fd8 000000000000fb88 ffff8810037185f8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] io_schedule+0x73/0xc0 2014-01-07 18:55:15 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:15 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:15 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:15 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:15 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:15 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:15 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:15 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:15 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:15 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:15 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:15 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:15 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:15 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:15 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:15 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:15 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:15 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:15 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:15 [] kthread+0x96/0xa0 2014-01-07 18:55:15 [] child_rip+0xa/0x20 2014-01-07 18:55:15 [] ? kthread+0x0/0xa0 2014-01-07 18:55:15 [] ? child_rip+0x0/0x20 2014-01-07 18:55:15 ll_ost_io02_0 S 0000000000000008 0 7438 2 0x00000000 2014-01-07 18:55:15 ffff88100373bda0 0000000000000046 0000000000000000 ffff88100373bd64 2014-01-07 18:55:15 ffff881000000000 ffff88083fe82000 ffff88085c476740 000000000000015a 2014-01-07 18:55:15 ffff881003739af8 ffff88100373bfd8 000000000000fb88 ffff881003739af8 2014-01-07 18:55:15 Call Trace: 2014-01-07 18:55:15 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io01_0 D 0000000000000004 0 7439 2 0x00000000 2014-01-07 18:55:16 ffff88100373f690 0000000000000046 0000000000000000 ffff88100373f654 2014-01-07 18:55:16 ffff881000000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:16 ffff881003739098 ffff88100373ffd8 000000000000fb88 ffff8807d1a44000 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:16 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:16 [] ? dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:16 [] ? dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:16 [] ? dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:16 [] ? osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:16 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:16 [] ? ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:16 [] ? ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ? obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:16 [] ? ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:16 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ? ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? kthread+0x96/0xa0 2014-01-07 18:55:16 [] ? child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io01_0 S 0000000000000004 0 7440 2 0x00000000 2014-01-07 18:55:16 ffff881003765930 0000000000000046 0000000000000000 ffff8810037658f4 2014-01-07 18:55:16 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:16 ffff881003738638 ffff881003765fd8 000000000000fb88 ffff881003738638 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:16 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:16 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:16 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io00_0 S 0000000000000003 0 7441 2 0x00000000 2014-01-07 18:55:16 ffff881003769930 0000000000000046 0000000000000000 ffff8810037698f4 2014-01-07 18:55:16 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:16 ffff881003767ab8 ffff881003769fd8 000000000000fb88 ffff881003767ab8 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:16 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:16 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:16 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io00_0 S 0000000000000003 0 7442 2 0x00000000 2014-01-07 18:55:16 ffff88100376d930 0000000000000046 0000000000000000 ffff88100376d8f4 2014-01-07 18:55:16 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:16 ffff881003767058 ffff88100376dfd8 000000000000fb88 ffff881003767058 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:16 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:16 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:16 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io03_0 S 000000000000000c 0 7443 2 0x00000000 2014-01-07 18:55:16 ffff88100376fda0 0000000000000046 0000000000000000 ffff88100376fd64 2014-01-07 18:55:16 ffff881000000000 ffff88083fe82800 ffff88085c476740 000000000000015a 2014-01-07 18:55:16 ffff8810037665f8 ffff88100376ffd8 000000000000fb88 ffff8810037665f8 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io01_0 S 0000000000000004 0 7444 2 0x00000000 2014-01-07 18:55:16 ffff881003779930 0000000000000046 0000000000000000 ffff8810037798f4 2014-01-07 18:55:16 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:16 ffff881003773af8 ffff881003779fd8 000000000000fb88 ffff881003773af8 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:16 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:16 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:16 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_server_post_idle_rqbds+0x38/0xe0 [ptlrpc] 2014-01-07 18:55:16 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io00_0 S 0000000000000000 0 7445 2 0x00000000 2014-01-07 18:55:16 ffff88100377d930 0000000000000046 ffff88100377d8f8 ffff88100377d8f4 2014-01-07 18:55:16 00052ccb00000000 0000000000000282 ffff88100377d8d0 ffffffff810810cc 2014-01-07 18:55:16 ffff881003773098 ffff88100377dfd8 000000000000fb88 ffff881003773098 2014-01-07 18:55:16 Call Trace: 2014-01-07 18:55:16 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:16 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:16 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:16 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:16 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:16 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:16 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:16 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:16 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:16 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:16 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:16 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:16 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:16 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:16 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:16 [] kthread+0x96/0xa0 2014-01-07 18:55:16 [] child_rip+0xa/0x20 2014-01-07 18:55:16 [] ? kthread+0x0/0xa0 2014-01-07 18:55:16 [] ? child_rip+0x0/0x20 2014-01-07 18:55:16 ll_ost_io00_0 S 0000000000000003 0 7446 2 0x00000000 2014-01-07 18:55:16 ffff88100377fda0 0000000000000046 ffff88100377fd68 ffff88100377fd64 2014-01-07 18:55:16 ffff880700000000 ffff88083fe81600 ffff880044656740 0000000000000400 2014-01-07 18:55:17 ffff881003772638 ffff88100377ffd8 000000000000fb88 ffff881003772638 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io01_0 D 0000000000000006 0 7447 2 0x00000000 2014-01-07 18:55:17 ffff881003793690 0000000000000046 ffff881003793658 ffff881003793654 2014-01-07 18:55:17 ffff8810037936b0 ffff88083fe81c00 ffff880044696740 0000000000000400 2014-01-07 18:55:17 ffff881003791ab8 ffff881003793fd8 000000000000fb88 ffff881003791ab8 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] io_schedule+0x73/0xc0 2014-01-07 18:55:17 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:17 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:17 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:17 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:17 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:17 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:17 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:17 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:17 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:17 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:17 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:17 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io01_0 S 0000000000000005 0 7448 2 0x00000000 2014-01-07 18:55:17 ffff881003797930 0000000000000046 0000000000000000 ffff8810037978f4 2014-01-07 18:55:17 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:17 ffff8810041b25f8 ffff881003797fd8 000000000000fb88 ffff8810041b25f8 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io00_0 S 0000000000000003 0 7449 2 0x00000000 2014-01-07 18:55:17 ffff881003799da0 0000000000000046 0000000000000000 ffff881003799d64 2014-01-07 18:55:17 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:17 ffff881003807058 ffff881003799fd8 000000000000fb88 ffff881003807058 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io00_0 S 0000000000000000 0 7450 2 0x00000000 2014-01-07 18:55:17 ffff88100379d930 0000000000000046 0000000000000000 ffff88100379d8f4 2014-01-07 18:55:17 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:17 ffff8810038065f8 ffff88100379dfd8 000000000000fb88 ffff8810038065f8 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io03_0 S 000000000000000e 0 7451 2 0x00000000 2014-01-07 18:55:17 ffff8810037a1930 0000000000000046 ffff8810037a18f8 ffff8810037a18f4 2014-01-07 18:55:17 52ccbe0f00000000 ffff88083fe82c00 ffff880044616740 0000000000000217 2014-01-07 18:55:17 ffff88100379faf8 ffff8810037a1fd8 000000000000fb88 ffff88100379faf8 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io03_0 S 000000000000000c 0 7452 2 0x00000000 2014-01-07 18:55:17 ffff8810037a9930 0000000000000046 ffff8810037a98f8 ffff8810037a98f4 2014-01-07 18:55:17 52ccbe1000000000 ffff88083fe82800 ffff880044616740 000000000000021f 2014-01-07 18:55:17 ffff88100379f098 ffff8810037a9fd8 000000000000fb88 ffff88100379f098 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io03_0 S 000000000000000c 0 7453 2 0x00000000 2014-01-07 18:55:17 ffff8810037ab930 0000000000000046 ffff8810037ab8f8 ffff8810037ab8f4 2014-01-07 18:55:17 52ccbe0e00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:17 ffff88100379e638 ffff8810037abfd8 000000000000fb88 ffff88100379e638 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:17 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:17 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:17 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:17 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:17 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:17 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:17 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:17 [] kthread+0x96/0xa0 2014-01-07 18:55:17 [] child_rip+0xa/0x20 2014-01-07 18:55:17 [] ? kthread+0x0/0xa0 2014-01-07 18:55:17 [] ? child_rip+0x0/0x20 2014-01-07 18:55:17 ll_ost_io01_0 S 0000000000000006 0 7454 2 0x00000000 2014-01-07 18:55:17 ffff8810037e7930 0000000000000046 0000000000000000 ffff8810037e78f4 2014-01-07 18:55:17 00052ccb00000000 0000000000000286 ffff8810037e78d0 ffffffff810810cc 2014-01-07 18:55:17 ffff8810037e5ab8 ffff8810037e7fd8 000000000000fb88 ffff8810037e5ab8 2014-01-07 18:55:17 Call Trace: 2014-01-07 18:55:17 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:17 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:17 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:17 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:17 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:17 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:17 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:17 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io01_0 D 0000000000000005 0 7455 2 0x00000000 2014-01-07 18:55:18 ffff881003009690 0000000000000046 ffff8810030096b0 ffffffffa04d9367 2014-01-07 18:55:18 ffff8810030096b0 0000000000000001 0000000000000ae5 ffff880e00000000 2014-01-07 18:55:18 ffff8810037e5058 ffff881003009fd8 000000000000fb88 ffff8810037e5058 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:18 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:18 [] io_schedule+0x73/0xc0 2014-01-07 18:55:18 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:18 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:18 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:18 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:18 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:18 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:18 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:18 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:18 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io00_0 S 0000000000000000 0 7456 2 0x00000000 2014-01-07 18:55:18 ffff88100300bda0 0000000000000046 0000000000000000 ffff88100300bd64 2014-01-07 18:55:18 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:18 ffff8810037e45f8 ffff88100300bfd8 000000000000fb88 ffff8810037e45f8 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:18 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io03_0 D 000000000000000e 0 7457 2 0x00000000 2014-01-07 18:55:18 ffff881003011360 0000000000000046 0000000000000000 ffff881003011324 2014-01-07 18:55:18 ffff881000000000 ffff88083fe82c00 ffff88085c476740 000000000000015a 2014-01-07 18:55:18 ffff88100300faf8 ffff881003011fd8 000000000000fb88 ffff88100300faf8 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] io_schedule+0x73/0xc0 2014-01-07 18:55:18 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:18 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:18 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:18 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:18 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:18 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:18 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:18 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:18 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:18 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:18 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:18 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:18 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:18 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:18 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:18 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:18 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io03_0 S 000000000000000c 0 7458 2 0x00000000 2014-01-07 18:55:18 ffff881003019930 0000000000000046 ffff8810030198f8 ffff8810030198f4 2014-01-07 18:55:18 52ccbdf800000000 0000000000000286 ffff8810030198d0 ffffffff810810cc 2014-01-07 18:55:18 ffff88100300f098 ffff881003019fd8 000000000000fb88 ffff88100300f098 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:18 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:18 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:18 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:18 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:18 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:18 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:18 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io01_0 S 0000000000000004 0 7459 2 0x00000000 2014-01-07 18:55:18 ffff88100301b930 0000000000000046 0000000000000000 ffff88100301b8f4 2014-01-07 18:55:18 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:18 ffff88100300e638 ffff88100301bfd8 000000000000fb88 ffff88100300e638 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:18 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:18 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:18 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:18 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:18 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:18 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:18 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:18 [] kthread+0x96/0xa0 2014-01-07 18:55:18 [] child_rip+0xa/0x20 2014-01-07 18:55:18 [] ? kthread+0x0/0xa0 2014-01-07 18:55:18 [] ? child_rip+0x0/0x20 2014-01-07 18:55:18 ll_ost_io02_0 D 000000000000000a 0 7460 2 0x00000000 2014-01-07 18:55:18 ffff881003043690 0000000000000046 ffff8810030436b0 ffffffffa04d9367 2014-01-07 18:55:18 ffff8810030436b0 0000000000000001 0000000000000e39 ffff880e00000000 2014-01-07 18:55:18 ffff881003041ab8 ffff881003043fd8 000000000000fb88 ffff881003041ab8 2014-01-07 18:55:18 Call Trace: 2014-01-07 18:55:18 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:18 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:18 [] io_schedule+0x73/0xc0 2014-01-07 18:55:18 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:18 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:18 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:18 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:18 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:18 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:18 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:18 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:18 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:18 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:18 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:18 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:18 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:18 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:18 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:18 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:18 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io02_0 S 0000000000000009 0 7461 2 0x00000000 2014-01-07 18:55:19 ffff881003077930 0000000000000046 ffff8810030778f8 ffff8810030778f4 2014-01-07 18:55:19 00052ccb00000000 ffff88083fe82200 ffff880044616740 0000000000000217 2014-01-07 18:55:19 ffff881003041058 ffff881003077fd8 000000000000fb88 ffff881003041058 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:19 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:19 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:19 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:19 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:19 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:19 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:19 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:19 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:19 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io01_0 D 0000000000000004 0 7462 2 0x00000000 2014-01-07 18:55:19 ffff881003091690 0000000000000046 ffff8810030916b0 ffffffffa04d97d6 2014-01-07 18:55:19 ffff8810030916b0 ffffffff00000001 ffff8806bf15a900 ffff881000000001 2014-01-07 18:55:19 ffff8810030405f8 ffff881003091fd8 000000000000fb88 ffff8810030405f8 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] ? dmu_zfetch+0x7c6/0xd70 [zfs] 2014-01-07 18:55:19 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:19 [] io_schedule+0x73/0xc0 2014-01-07 18:55:19 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:19 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:19 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:19 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:19 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:19 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:19 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:19 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:19 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:19 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:19 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:19 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io00_0 D 0000000000000002 0 7463 2 0x00000000 2014-01-07 18:55:19 ffff881003095690 0000000000000046 0000000000000000 ffff881003095654 2014-01-07 18:55:19 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:19 ffff881003093af8 ffff881003095fd8 000000000000fb88 ffff881003093af8 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] io_schedule+0x73/0xc0 2014-01-07 18:55:19 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:19 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:19 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:19 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:19 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:19 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:19 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:19 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:19 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:19 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:19 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:19 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io00_0 D 0000000000000002 0 7464 2 0x00000000 2014-01-07 18:55:19 ffff88100309b690 0000000000000046 0000000000000000 ffff88100309b654 2014-01-07 18:55:19 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:19 ffff881003093098 ffff88100309bfd8 000000000000fb88 ffff881003093098 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] io_schedule+0x73/0xc0 2014-01-07 18:55:19 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:19 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:19 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:19 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:19 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:19 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:19 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:19 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:19 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:19 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:19 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:19 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:19 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io02_0 S 000000000000000a 0 7465 2 0x00000000 2014-01-07 18:55:19 ffff88100309dda0 0000000000000046 0000000000000000 ffff88100309dd64 2014-01-07 18:55:19 ffff881000000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:55:19 ffff881003092638 ffff88100309dfd8 000000000000fb88 ffff881003092638 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:19 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io00_0 S 0000000000000002 0 7466 2 0x00000000 2014-01-07 18:55:19 ffff8810030b5930 0000000000000046 0000000000000000 ffff8810030b58f4 2014-01-07 18:55:19 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:19 ffff88100309fab8 ffff8810030b5fd8 000000000000fb88 ffff88100309fab8 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:19 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:19 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:19 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:19 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:19 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:19 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:19 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:19 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:19 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:19 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:19 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:19 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:19 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:19 [] kthread+0x96/0xa0 2014-01-07 18:55:19 [] child_rip+0xa/0x20 2014-01-07 18:55:19 [] ? kthread+0x0/0xa0 2014-01-07 18:55:19 [] ? child_rip+0x0/0x20 2014-01-07 18:55:19 ll_ost_io00_0 S 0000000000000003 0 7467 2 0x00000000 2014-01-07 18:55:19 ffff8810030b9930 0000000000000046 ffff8810030b98f8 ffff8810030b98f4 2014-01-07 18:55:19 00052ccb00000000 ffff88083fe81600 ffff880044616740 0000000000000207 2014-01-07 18:55:19 ffff88100309f058 ffff8810030b9fd8 000000000000fb88 ffff88100309f058 2014-01-07 18:55:19 Call Trace: 2014-01-07 18:55:19 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:19 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:19 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:19 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:19 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:20 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:20 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 S 0000000000000004 0 7468 2 0x00000000 2014-01-07 18:55:20 ffff8810030bfda0 0000000000000046 0000000000000000 ffff8810030bfd64 2014-01-07 18:55:20 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff88100309e5f8 ffff8810030bffd8 000000000000fb88 ffff88100309e5f8 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:20 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 S 0000000000000004 0 7469 2 0x00000000 2014-01-07 18:55:20 ffff8810030c3930 0000000000000046 0000000000000000 ffff8810030c38f4 2014-01-07 18:55:20 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff8810030c1af8 ffff8810030c3fd8 000000000000fb88 ffff8810030c1af8 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:20 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:20 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:20 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:20 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:20 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:20 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 D 0000000000000006 0 7470 2 0x00000000 2014-01-07 18:55:20 ffff8810030c7690 0000000000000046 0000000000000000 ffff8810030c7654 2014-01-07 18:55:20 ffff881000000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff8810030c1098 ffff8810030c7fd8 000000000000fb88 ffff8810030c1098 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] io_schedule+0x73/0xc0 2014-01-07 18:55:20 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:20 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:20 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:20 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:20 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:20 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:20 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:20 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:20 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 S 0000000000000004 0 7471 2 0x00000000 2014-01-07 18:55:20 ffff8810030cb930 0000000000000046 0000000000000000 ffff8810030cb8f4 2014-01-07 18:55:20 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff8810030c0638 ffff8810030cbfd8 000000000000fb88 ffff8810030c0638 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:20 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:20 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:20 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:20 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:20 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:20 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 S 0000000000000005 0 7472 2 0x00000000 2014-01-07 18:55:20 ffff8810030cf930 0000000000000046 0000000000000000 ffff8810030cf8f4 2014-01-07 18:55:20 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff8810030cdab8 ffff8810030cffd8 000000000000fb88 ffff8810030cdab8 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:20 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:20 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:20 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:20 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:20 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:20 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 D 0000000000000004 0 7473 2 0x00000000 2014-01-07 18:55:20 ffff8810030d3690 0000000000000046 ffff8810030d36b0 ffffffffa04d9367 2014-01-07 18:55:20 ffff8810030d36b0 0000000000000001 0000000000000bfe ffff880e00000000 2014-01-07 18:55:20 ffff8810030cd058 ffff8810030d3fd8 000000000000fb88 ffff8810030cd058 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:20 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:20 [] io_schedule+0x73/0xc0 2014-01-07 18:55:20 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:20 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:20 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:20 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:20 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:20 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:20 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:20 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:20 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:20 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:20 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:20 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:20 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:20 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:20 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:20 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:20 [] kthread+0x96/0xa0 2014-01-07 18:55:20 [] child_rip+0xa/0x20 2014-01-07 18:55:20 [] ? kthread+0x0/0xa0 2014-01-07 18:55:20 [] ? child_rip+0x0/0x20 2014-01-07 18:55:20 ll_ost_io01_0 D 0000000000000005 0 7474 2 0x00000000 2014-01-07 18:55:20 ffff8810030d5690 0000000000000046 0000000000000000 ffff8810030d5654 2014-01-07 18:55:20 ffff881000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:20 ffff8810030cc5f8 ffff8810030d5fd8 000000000000fb88 ffff8810030cc5f8 2014-01-07 18:55:20 Call Trace: 2014-01-07 18:55:20 [] io_schedule+0x73/0xc0 2014-01-07 18:55:20 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:20 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:20 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:20 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:20 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:20 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:21 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:21 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:21 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:21 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:21 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io03_0 S 000000000000000d 0 7475 2 0x00000000 2014-01-07 18:55:21 ffff8810030d9930 0000000000000046 ffff8810030d98f8 ffff8810030d98f4 2014-01-07 18:55:21 52ccbdf800000000 ffff88083fe82a00 ffff880044616740 0000000000000416 2014-01-07 18:55:21 ffff8810030d7af8 ffff8810030d9fd8 000000000000fb88 ffff8810030d7af8 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:21 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:21 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:21 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:21 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io02_0 S 000000000000000a 0 7476 2 0x00000000 2014-01-07 18:55:21 ffff8810030ddda0 0000000000000046 0000000000000000 ffff8810030ddd64 2014-01-07 18:55:21 ffff881000000000 ffff88083fe82400 ffff88085c476740 000000000000015a 2014-01-07 18:55:21 ffff8810030d7098 ffff8810030ddfd8 000000000000fb88 ffff8810030d7098 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io02_0 S 0000000000000009 0 7477 2 0x00000000 2014-01-07 18:55:21 ffff8810030dfda0 0000000000000046 0000000000000000 ffff881003ba49c0 2014-01-07 18:55:21 ffff881031875c00 0000000000000000 000000000000000f ffff8807d1a359c0 2014-01-07 18:55:21 ffff8810030d6638 ffff8810030dffd8 000000000000fb88 ffff8810030d6638 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io03_0 D 000000000000000d 0 7478 2 0x00000000 2014-01-07 18:55:21 ffff8810030ff690 0000000000000046 ffff8810030ff6b0 ffffffffa04d9367 2014-01-07 18:55:21 ffff8810030ff6b0 0000000000000001 0000000000000cf3 ffff880e00000000 2014-01-07 18:55:21 ffff8810030fdab8 ffff8810030fffd8 000000000000fb88 ffff8810030fdab8 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:21 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:21 [] io_schedule+0x73/0xc0 2014-01-07 18:55:21 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:21 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:21 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:21 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:21 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:21 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:21 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:21 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:21 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:21 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:21 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io00_0 S 0000000000000000 0 7479 2 0x00000000 2014-01-07 18:55:21 ffff881003147930 0000000000000046 0000000000000000 ffff8810031478f4 2014-01-07 18:55:21 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:21 ffff8810030fd058 ffff881003147fd8 000000000000fb88 ffff8810030fd058 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:21 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:21 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:21 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:21 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io03_0 S 000000000000000f 0 7480 2 0x00000000 2014-01-07 18:55:21 ffff881003149930 0000000000000046 ffff8810031498f8 ffff8810031498f4 2014-01-07 18:55:21 52ccbdfc00000000 ffff88083fe82e00 ffff880044616740 000000000000021f 2014-01-07 18:55:21 ffff881003791058 ffff881003149fd8 000000000000fb88 ffff881003791058 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:21 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:21 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:21 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:21 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io00_0 S 0000000000000000 0 7481 2 0x00000000 2014-01-07 18:55:21 ffff88100314b930 0000000000000046 ffff88100314b8f8 ffff88100314b8f4 2014-01-07 18:55:21 00052ccb00000000 0000000000000282 ffff88100314b8d0 ffffffff810810cc 2014-01-07 18:55:21 ffff8810037905f8 ffff88100314bfd8 000000000000fb88 ffff8810037905f8 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:21 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:21 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:21 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:21 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:21 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:21 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:21 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:21 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:21 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:21 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:21 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:21 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:21 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:21 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:21 [] kthread+0x96/0xa0 2014-01-07 18:55:21 [] child_rip+0xa/0x20 2014-01-07 18:55:21 [] ? kthread+0x0/0xa0 2014-01-07 18:55:21 [] ? child_rip+0x0/0x20 2014-01-07 18:55:21 ll_ost_io00_0 D 0000000000000002 0 7482 2 0x00000000 2014-01-07 18:55:21 ffff881003163690 0000000000000046 0000000000000000 ffff881003163654 2014-01-07 18:55:21 ffff881000000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:21 ffff881003161af8 ffff881003163fd8 000000000000fb88 ffff881003161af8 2014-01-07 18:55:21 Call Trace: 2014-01-07 18:55:21 [] io_schedule+0x73/0xc0 2014-01-07 18:55:21 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:21 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:21 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:21 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:22 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:22 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:22 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:22 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:22 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:22 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io02_0 D 0000000000000009 0 7483 2 0x00000000 2014-01-07 18:55:22 ffff88100316d690 0000000000000046 ffff88100316d658 ffff88100316d654 2014-01-07 18:55:22 ffff881000000000 ffff88083fe82200 ffff880044616740 0000000000000217 2014-01-07 18:55:22 ffff8810042b2638 ffff88100316dfd8 000000000000fb88 ffff8810042b2638 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] io_schedule+0x73/0xc0 2014-01-07 18:55:22 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:22 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:22 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:22 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:22 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:22 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:22 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:22 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:22 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io01_0 D 0000000000000005 0 7484 2 0x00000000 2014-01-07 18:55:22 ffff881003189690 0000000000000046 ffff881003189658 ffff881003189654 2014-01-07 18:55:22 ffff8810031896b0 ffff88083fe81a00 ffff880044616740 0000000000000207 2014-01-07 18:55:22 ffff881003187ab8 ffff881003189fd8 000000000000fb88 ffff881003187ab8 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] io_schedule+0x73/0xc0 2014-01-07 18:55:22 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:22 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:22 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:22 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:22 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:22 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:22 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:22 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:22 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:22 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:22 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io01_0 S 0000000000000007 0 7485 2 0x00000000 2014-01-07 18:55:22 ffff88100318b930 0000000000000046 0000000000000000 ffff88100318b8f4 2014-01-07 18:55:22 00052ccb00000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:22 ffff881003187058 ffff88100318bfd8 000000000000fb88 ffff881003187058 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:22 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:22 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:22 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:22 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:22 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:22 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io00_0 S 0000000000000002 0 7486 2 0x00000000 2014-01-07 18:55:22 ffff88100318f930 0000000000000046 ffff88100318f8f8 ffff88100318f8f4 2014-01-07 18:55:22 0000000000000000 ffff88083fe81400 ffff88085c476740 0000000000000269 2014-01-07 18:55:22 ffff8810031865f8 ffff88100318ffd8 000000000000fb88 ffff8810031865f8 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:22 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:22 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:22 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:22 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:22 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:22 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io01_0 S 0000000000000006 0 7487 2 0x00000000 2014-01-07 18:55:22 ffff881003193da0 0000000000000046 0000000000000000 ffff881003193d64 2014-01-07 18:55:22 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:22 ffff881003191af8 ffff881003193fd8 000000000000fb88 ffff881003191af8 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:22 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io01_0 S 0000000000000005 0 7488 2 0x00000000 2014-01-07 18:55:22 ffff881003195930 0000000000000046 0000000000000000 ffff8810031958f4 2014-01-07 18:55:22 00052ccb00000000 0000000000000286 ffff8810031958d0 ffffffff810810cc 2014-01-07 18:55:22 ffff881003191098 ffff881003195fd8 000000000000fb88 ffff881003191098 2014-01-07 18:55:22 Call Trace: 2014-01-07 18:55:22 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:22 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:22 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:22 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:22 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:22 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:22 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:22 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:22 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:22 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:22 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:22 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:22 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:22 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:22 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:22 [] kthread+0x96/0xa0 2014-01-07 18:55:22 [] child_rip+0xa/0x20 2014-01-07 18:55:22 [] ? kthread+0x0/0xa0 2014-01-07 18:55:22 [] ? child_rip+0x0/0x20 2014-01-07 18:55:22 ll_ost_io01_0 D 0000000000000005 0 7489 2 0x00000000 2014-01-07 18:55:22 ffff881003199690 0000000000000046 0000000000000000 ffff881003199654 2014-01-07 18:55:23 ffff881000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:23 ffff881003190638 ffff881003199fd8 000000000000fb88 ffff881003190638 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] io_schedule+0x73/0xc0 2014-01-07 18:55:23 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:23 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:23 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:23 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:23 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:23 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:23 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:23 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:23 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:23 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:23 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:23 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io03_0 S 000000000000000d 0 7490 2 0x00000000 2014-01-07 18:55:23 ffff88100319f930 0000000000000046 ffff88100319f8f8 ffff88100319f8f4 2014-01-07 18:55:23 52ccbe0f00000000 0000000000000286 ffff88100319f8d0 ffffffff810810cc 2014-01-07 18:55:23 ffff88100319dab8 ffff88100319ffd8 000000000000fb88 ffff88100319dab8 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:23 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:23 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:23 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:23 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:23 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:23 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:23 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:23 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:23 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:23 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io00_0 S 0000000000000003 0 7491 2 0x00000000 2014-01-07 18:55:23 ffff8810031b5da0 0000000000000046 0000000000000000 ffff8810031b5d64 2014-01-07 18:55:23 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:23 ffff88100319d058 ffff8810031b5fd8 000000000000fb88 ffff88100319d058 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:23 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io02_0 S 0000000000000009 0 7492 2 0x00000000 2014-01-07 18:55:23 ffff8810031b9da0 0000000000000046 0000000000000010 ffff881003ba4780 2014-01-07 18:55:23 ffff881031875c00 0000000000000000 0000000000000042 ffff8807d1a359c0 2014-01-07 18:55:23 ffff88100319c5f8 ffff8810031b9fd8 000000000000fb88 ffff88100319c5f8 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:23 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:23 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:23 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io01_0 S 0000000000000004 0 7493 2 0x00000000 2014-01-07 18:55:23 ffff8810031d1930 0000000000000046 0000000000000000 ffff8810031d18f4 2014-01-07 18:55:23 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:23 ffff8810031bbaf8 ffff8810031d1fd8 000000000000fb88 ffff8810031bbaf8 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:23 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:23 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:23 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:23 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:23 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:23 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:23 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:23 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:23 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io00_0 D 0000000000000002 0 7494 2 0x00000000 2014-01-07 18:55:23 ffff8810031d3690 0000000000000046 ffff8810031d36b0 ffffffffa04d9367 2014-01-07 18:55:23 ffff8810031d36b0 0000000000000001 0000000000000ce9 ffff880e00000000 2014-01-07 18:55:23 ffff8810031bb098 ffff8810031d3fd8 000000000000fb88 ffff8810031bb098 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:23 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:23 [] io_schedule+0x73/0xc0 2014-01-07 18:55:23 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:23 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:23 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:23 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:23 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:23 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:23 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:23 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:23 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:23 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:23 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:23 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:23 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:23 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:23 [] kthread+0x96/0xa0 2014-01-07 18:55:23 [] child_rip+0xa/0x20 2014-01-07 18:55:23 [] ? kthread+0x0/0xa0 2014-01-07 18:55:23 [] ? child_rip+0x0/0x20 2014-01-07 18:55:23 ll_ost_io03_0 D 000000000000000d 0 7495 2 0x00000000 2014-01-07 18:55:23 ffff8810031d7360 0000000000000046 0000000000000000 ffff8810031d7324 2014-01-07 18:55:23 ffff881000000000 ffff88083fe82a00 ffff88085c476740 000000000000015a 2014-01-07 18:55:23 ffff8810031ba638 ffff8810031d7fd8 000000000000fb88 ffff8810031ba638 2014-01-07 18:55:23 Call Trace: 2014-01-07 18:55:23 [] io_schedule+0x73/0xc0 2014-01-07 18:55:23 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:23 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:23 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:23 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:23 [] dbuf_read+0x3fd/0x750 [zfs] 2014-01-07 18:55:23 [] ? __dbuf_hold_impl+0x2b1/0x480 [zfs] 2014-01-07 18:55:23 [] dbuf_prefetch+0x10b/0x270 [zfs] 2014-01-07 18:55:23 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:23 [] dmu_zfetch_dofetch+0xf1/0x160 [zfs] 2014-01-07 18:55:23 [] dmu_zfetch+0x80c/0xd70 [zfs] 2014-01-07 18:55:23 [] ? arc_read+0x6a9/0x8a0 [zfs] 2014-01-07 18:55:23 [] dbuf_read+0x48a/0x750 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_dnode+0x162/0x560 [zfs] 2014-01-07 18:55:23 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:23 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:23 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:23 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:23 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:23 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:23 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:23 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:23 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:23 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:23 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:23 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:23 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io02_0 S 0000000000000008 0 7496 2 0x00000000 2014-01-07 18:55:24 ffff88100320dda0 0000000000000046 ffff88100320dd68 ffff88100320dd64 2014-01-07 18:55:24 ffff881031875c00 ffff88083fe82000 ffff880044636740 0000000000000207 2014-01-07 18:55:24 ffff88100320bab8 ffff88100320dfd8 000000000000fb88 ffff88100320bab8 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io00_0 S 0000000000000002 0 7497 2 0x00000000 2014-01-07 18:55:24 ffff881003225930 0000000000000046 0000000000000000 ffff8810032258f4 2014-01-07 18:55:24 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff88100320b058 ffff881003225fd8 000000000000fb88 ffff88100320b058 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io02_0 S 0000000000000008 0 7498 2 0x00000000 2014-01-07 18:55:24 ffff881003229da0 0000000000000046 0000000000000000 ffff881003229d64 2014-01-07 18:55:24 ffff881000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:55:24 ffff88100320a5f8 ffff881003229fd8 000000000000fb88 ffff88100320a5f8 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io00_0 S 0000000000000003 0 7499 2 0x00000000 2014-01-07 18:55:24 ffff881003241930 0000000000000046 0000000000000000 ffff8810032418f4 2014-01-07 18:55:24 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff88100322baf8 ffff881003241fd8 000000000000fb88 ffff88100322baf8 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io01_0 S 0000000000000004 0 7500 2 0x00000000 2014-01-07 18:55:24 ffff881003243930 0000000000000046 0000000000000000 ffff8810032438f4 2014-01-07 18:55:24 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff88100322b098 ffff881003243fd8 000000000000fb88 ffff88100322b098 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io01_0 S 0000000000000004 0 7501 2 0x00000000 2014-01-07 18:55:24 ffff881003247930 0000000000000046 ffff8810032478f8 ffff8810032478f4 2014-01-07 18:55:24 00052ccb00000000 ffff88083fe81800 ffff88085c476740 0000000000000269 2014-01-07 18:55:24 ffff88100322a638 ffff881003247fd8 000000000000fb88 ffff88100322a638 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io01_0 S 0000000000000004 0 7502 2 0x00000000 2014-01-07 18:55:24 ffff88100324b930 0000000000000046 0000000000000000 ffff88100324b8f4 2014-01-07 18:55:24 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff881003249ab8 ffff88100324bfd8 000000000000fb88 ffff881003249ab8 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io01_0 S 0000000000000005 0 7503 2 0x00000000 2014-01-07 18:55:24 ffff88100324dda0 0000000000000046 0000000000000000 ffff88100324dd64 2014-01-07 18:55:24 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff881003249058 ffff88100324dfd8 000000000000fb88 ffff881003249058 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] kthread+0x96/0xa0 2014-01-07 18:55:24 [] child_rip+0xa/0x20 2014-01-07 18:55:24 [] ? kthread+0x0/0xa0 2014-01-07 18:55:24 [] ? child_rip+0x0/0x20 2014-01-07 18:55:24 ll_ost_io01_0 S 0000000000000005 0 7504 2 0x00000000 2014-01-07 18:55:24 ffff881003253930 0000000000000046 0000000000000000 ffff8810032538f4 2014-01-07 18:55:24 0000000000000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:24 ffff8810032485f8 ffff881003253fd8 000000000000fb88 ffff8810032485f8 2014-01-07 18:55:24 Call Trace: 2014-01-07 18:55:24 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:24 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:24 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:24 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:24 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:24 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:24 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:24 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:24 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:24 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:24 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:24 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:24 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:24 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:24 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io00_0 S 0000000000000002 0 7505 2 0x00000000 2014-01-07 18:55:25 ffff881003257930 0000000000000046 0000000000000000 ffff8810032578f4 2014-01-07 18:55:25 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:25 ffff881003255af8 ffff881003257fd8 000000000000fb88 ffff881003255af8 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:25 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:25 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:25 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:25 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:25 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:25 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:25 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io01_0 S 0000000000000007 0 7506 2 0x00000000 2014-01-07 18:55:25 ffff881003259da0 0000000000000046 0000000000000000 ffff881003259d64 2014-01-07 18:55:25 ffff880700000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:25 ffff881003255098 ffff881003259fd8 000000000000fb88 ffff881003255098 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io03_0 D 000000000000000c 0 7507 2 0x00000000 2014-01-07 18:55:25 ffff88100327d690 0000000000000046 ffff88100327d658 ffff88100327d654 2014-01-07 18:55:25 ffff881000000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:25 ffff881003254638 ffff88100327dfd8 000000000000fb88 ffff881003254638 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] io_schedule+0x73/0xc0 2014-01-07 18:55:25 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:25 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:25 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:25 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:25 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:25 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:25 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:25 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:25 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:25 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:25 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:25 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:25 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:25 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io01_0 S 0000000000000004 0 7508 2 0x00000000 2014-01-07 18:55:25 ffff881003295da0 0000000000000046 0000000000000000 ffff881003295d64 2014-01-07 18:55:25 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:25 ffff88100327fab8 ffff881003295fd8 000000000000fb88 ffff88100327fab8 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io00_0 S 0000000000000002 0 7509 2 0x00000000 2014-01-07 18:55:25 ffff8810032b9930 0000000000000046 0000000000000001 ffff8807bba1f840 2014-01-07 18:55:25 00052ccb7753ed9d 0000000000000000 ffff8810032b9900 ffffffffa03847b1 2014-01-07 18:55:25 ffff88100327f058 ffff8810032b9fd8 000000000000fb88 ffff88100327f058 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:25 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:25 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:25 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:25 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:25 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:25 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:25 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io02_0 D 0000000000000008 0 7510 2 0x00000000 2014-01-07 18:55:25 ffff8810032bd690 0000000000000046 0000000000000000 ffff8810032bd654 2014-01-07 18:55:25 ffff881000000000 ffff88083fe82000 ffff88085c476740 000000000000015a 2014-01-07 18:55:25 ffff88100327e5f8 ffff8810032bdfd8 000000000000fb88 ffff88100327e5f8 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] io_schedule+0x73/0xc0 2014-01-07 18:55:25 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:25 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:25 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:25 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:25 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:25 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:25 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:25 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:25 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:25 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:25 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:25 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:25 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:25 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:25 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io01_0 S 0000000000000006 0 7511 2 0x00000000 2014-01-07 18:55:25 ffff8810032d7da0 0000000000000046 0000000000000000 ffff8810032d7d64 2014-01-07 18:55:25 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:25 ffff8810032d5af8 ffff8810032d7fd8 000000000000fb88 ffff8810032d5af8 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:25 [] kthread+0x96/0xa0 2014-01-07 18:55:25 [] child_rip+0xa/0x20 2014-01-07 18:55:25 [] ? kthread+0x0/0xa0 2014-01-07 18:55:25 [] ? child_rip+0x0/0x20 2014-01-07 18:55:25 ll_ost_io03_0 S 000000000000000c 0 7512 2 0x00000000 2014-01-07 18:55:25 ffff8810032d9930 0000000000000046 ffff8810032d98f8 ffff8810032d98f4 2014-01-07 18:55:25 52ccbdf800000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:25 ffff8810032d5098 ffff8810032d9fd8 000000000000fb88 ffff8810032d5098 2014-01-07 18:55:25 Call Trace: 2014-01-07 18:55:25 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:25 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:25 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:25 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:25 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:25 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:25 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:25 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:25 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:25 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:25 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:25 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:25 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:25 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io03_0 S 000000000000000d 0 7513 2 0x00000000 2014-01-07 18:55:26 ffff8810032f3930 0000000000000046 0000000000000000 ffff8810032f38f4 2014-01-07 18:55:26 52ccbe1000000000 ffff88083fe82a00 ffff88085c476740 000000000000015a 2014-01-07 18:55:26 ffff8810032d4638 ffff8810032f3fd8 000000000000fb88 ffff8810032d4638 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io01_0 S 0000000000000005 0 7514 2 0x00000000 2014-01-07 18:55:26 ffff881003311930 0000000000000046 0000000000000000 ffff8810033118f4 2014-01-07 18:55:26 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:26 ffff8810032ffab8 ffff881003311fd8 000000000000fb88 ffff8810032ffab8 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io00_0 S 0000000000000002 0 7515 2 0x00000000 2014-01-07 18:55:26 ffff881003313930 0000000000000046 ffff8810033138f8 ffff8810033138f4 2014-01-07 18:55:26 00052ccb00000000 ffff88083fe81400 ffff880044676740 0000000000000400 2014-01-07 18:55:26 ffff8810032ff058 ffff881003313fd8 000000000000fb88 ffff8810032ff058 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io03_0 S 000000000000000e 0 7516 2 0x00000000 2014-01-07 18:55:26 ffff881003317930 0000000000000046 ffff8810033178f8 ffff8810033178f4 2014-01-07 18:55:26 52ccbe0b00000000 ffff88083fe82c00 ffff880044616740 000000000000020f 2014-01-07 18:55:26 ffff8810032fe5f8 ffff881003317fd8 000000000000fb88 ffff8810032fe5f8 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io01_0 S 0000000000000006 0 7517 2 0x00000000 2014-01-07 18:55:26 ffff88100331fda0 0000000000000046 0000000000000000 ffff88100331fd64 2014-01-07 18:55:26 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:26 ffff88100331daf8 ffff88100331ffd8 000000000000fb88 ffff88100331daf8 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io01_0 S 0000000000000005 0 7518 2 0x00000000 2014-01-07 18:55:26 ffff881003333930 0000000000000046 0000000000000000 ffff8810033338f4 2014-01-07 18:55:26 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:26 ffff88100331d098 ffff881003333fd8 000000000000fb88 ffff88100331d098 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io01_0 S 0000000000000006 0 7519 2 0x00000000 2014-01-07 18:55:26 ffff881003337da0 0000000000000046 0000000000000000 ffff881003337d64 2014-01-07 18:55:26 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:26 ffff88100331c638 ffff881003337fd8 000000000000fb88 ffff88100331c638 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io03_0 S 000000000000000d 0 7520 2 0x00000000 2014-01-07 18:55:26 ffff88100333b930 0000000000000046 ffff88100333b8f8 ffff88100333b8f4 2014-01-07 18:55:26 52ccbe0c00000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:26 ffff881003339ab8 ffff88100333bfd8 000000000000fb88 ffff881003339ab8 2014-01-07 18:55:26 Call Trace: 2014-01-07 18:55:26 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:26 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:26 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:26 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:26 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:26 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:26 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:26 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:26 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:26 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:26 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:26 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:26 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:26 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:26 [] kthread+0x96/0xa0 2014-01-07 18:55:26 [] child_rip+0xa/0x20 2014-01-07 18:55:26 [] ? kthread+0x0/0xa0 2014-01-07 18:55:26 [] ? child_rip+0x0/0x20 2014-01-07 18:55:26 ll_ost_io03_0 S 000000000000000d 0 7521 2 0x00000000 2014-01-07 18:55:26 ffff881003351930 0000000000000046 ffff8810033518f8 ffff8810033518f4 2014-01-07 18:55:26 52ccbe0f00000000 0000000000000286 ffff8810033518d0 ffffffff810810cc 2014-01-07 18:55:27 ffff881003339058 ffff881003351fd8 000000000000fb88 ffff881003339058 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io03_0 S 000000000000000d 0 7522 2 0x00000000 2014-01-07 18:55:27 ffff88100335d930 0000000000000046 ffff88100335d8f8 ffff88100335d8f4 2014-01-07 18:55:27 52ccbe0f00000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:27 ffff8810033385f8 ffff88100335dfd8 000000000000fb88 ffff8810033385f8 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io03_0 S 000000000000000c 0 7523 2 0x00000000 2014-01-07 18:55:27 ffff881003389da0 0000000000000046 0000000000000010 ffff8810034e3380 2014-01-07 18:55:27 ffff8810065e4000 0000000000000000 0000000000000055 ffff8807d1a359c0 2014-01-07 18:55:27 ffff881003387af8 ffff881003389fd8 000000000000fb88 ffff881003387af8 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:27 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io03_0 S 000000000000000d 0 7524 2 0x00000000 2014-01-07 18:55:27 ffff8810033c3930 0000000000000046 000000000000006e 0000000100010000 2014-01-07 18:55:27 52ccbe0e0000000d 0000000000000286 ffff8810033c38d0 ffffffff810810cc 2014-01-07 18:55:27 ffff881003387098 ffff8810033c3fd8 000000000000fb88 ffff881003387098 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io03_0 S 000000000000000d 0 7525 2 0x00000000 2014-01-07 18:55:27 ffff8810033e1930 0000000000000046 0000000000000000 ffff8810033e18f4 2014-01-07 18:55:27 52ccbe0900000000 ffff88083fe82a00 ffff88085c476740 000000000000015a 2014-01-07 18:55:27 ffff881003386638 ffff8810033e1fd8 000000000000fb88 ffff881003386638 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io01_0 S 0000000000000004 0 7526 2 0x00000000 2014-01-07 18:55:27 ffff8810033e9930 0000000000000046 0000000000000000 ffff8810033e98f4 2014-01-07 18:55:27 00052ccb00000000 0000000000000286 ffff8810033e98d0 ffffffff810810cc 2014-01-07 18:55:27 ffff8810033e3ab8 ffff8810033e9fd8 000000000000fb88 ffff8810033e3ab8 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io01_0 S 0000000000000004 0 7527 2 0x00000000 2014-01-07 18:55:27 ffff8810033ebda0 0000000000000046 0000000000000000 ffff8810033ebd64 2014-01-07 18:55:27 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:27 ffff8810033e3058 ffff8810033ebfd8 000000000000fb88 ffff8810033e3058 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io01_0 S 0000000000000006 0 7528 2 0x00000000 2014-01-07 18:55:27 ffff880753769930 0000000000000046 0000000000000000 ffff8807537698f4 2014-01-07 18:55:27 00052ccb00000000 0000000000000286 ffff8807537698d0 ffffffff810810cc 2014-01-07 18:55:27 ffff8808319e9af8 ffff880753769fd8 000000000000fb88 ffff8808319e9af8 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:27 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:27 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:27 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:27 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:27 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:27 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:27 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:27 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:27 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:27 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:27 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:27 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:27 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:27 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:27 [] kthread+0x96/0xa0 2014-01-07 18:55:27 [] child_rip+0xa/0x20 2014-01-07 18:55:27 [] ? kthread+0x0/0xa0 2014-01-07 18:55:27 [] ? child_rip+0x0/0x20 2014-01-07 18:55:27 ll_ost_io00_0 S 0000000000000003 0 7529 2 0x00000000 2014-01-07 18:55:27 ffff88075334bda0 0000000000000046 0000000000000000 ffff88075334bd64 2014-01-07 18:55:27 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:27 ffff880833fcfaf8 ffff88075334bfd8 000000000000fb88 ffff880833fcfaf8 2014-01-07 18:55:27 Call Trace: 2014-01-07 18:55:27 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io03_0 S 000000000000000e 0 7530 2 0x00000000 2014-01-07 18:55:28 ffff88075336d930 0000000000000046 ffff88075336d8f8 ffff88075336d8f4 2014-01-07 18:55:28 52ccbdf800000000 ffff88083fe82c00 ffff880044616740 0000000000000217 2014-01-07 18:55:28 ffff880832f97ab8 ffff88075336dfd8 000000000000fb88 ffff880832f97ab8 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:28 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:28 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:28 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:28 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:28 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:28 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:28 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io02_0 D 0000000000000008 0 7531 2 0x00000000 2014-01-07 18:55:28 ffff88075336f690 0000000000000046 ffff88075336f6b0 ffffffffa04d9367 2014-01-07 18:55:28 ffff88075336f6b0 0000000000000001 0000000000000bb0 ffff880700000000 2014-01-07 18:55:28 ffff880832f965f8 ffff88075336ffd8 000000000000fb88 ffff880832f965f8 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:28 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:28 [] io_schedule+0x73/0xc0 2014-01-07 18:55:28 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:28 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:28 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:28 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:28 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:28 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:28 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:28 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:28 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:28 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:28 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:28 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:28 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:28 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io02_0 S 000000000000000a 0 7532 2 0x00000000 2014-01-07 18:55:28 ffff8810033efda0 0000000000000046 0000000000000000 ffff8810033efd64 2014-01-07 18:55:28 ffff881000000000 ffff88083fe82400 ffff88085c476740 0000000000000182 2014-01-07 18:55:28 ffff8810036e3098 ffff8810033effd8 000000000000fb88 ffff8810036e3098 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io01_0 S 0000000000000006 0 7533 2 0x00000000 2014-01-07 18:55:28 ffff881002c1b930 0000000000000046 0000000000000000 ffff881002c1b8f4 2014-01-07 18:55:28 00052ccb00000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:28 ffff8810036e2638 ffff881002c1bfd8 000000000000fb88 ffff8810036e2638 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:28 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:28 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:28 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:28 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:28 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:28 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:28 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io02_0 S 000000000000000a 0 7534 2 0x00000000 2014-01-07 18:55:28 ffff881002c41da0 0000000000000046 ffff881002c41d68 ffff881002c41d64 2014-01-07 18:55:28 ffff881031875c00 ffff88083fe82400 ffff880044636740 0000000000000207 2014-01-07 18:55:28 ffff8810030fc5f8 ffff881002c41fd8 000000000000fb88 ffff8810030fc5f8 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io00_0 S 0000000000000003 0 7535 2 0x00000000 2014-01-07 18:55:28 ffff881002c45da0 0000000000000046 0000000000000010 ffff8807b4603200 2014-01-07 18:55:28 ffff8807c0202400 0000000000000000 0000000000000038 ffff8807d1a359c0 2014-01-07 18:55:28 ffff881002c43af8 ffff881002c45fd8 000000000000fb88 ffff881002c43af8 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:28 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:28 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io01_0 S 0000000000000005 0 7536 2 0x00000000 2014-01-07 18:55:28 ffff881002c4dda0 0000000000000046 0000000000000000 ffff881002c4dd64 2014-01-07 18:55:28 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:28 ffff881002c43098 ffff881002c4dfd8 000000000000fb88 ffff881002c43098 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io02_0 D 0000000000000008 0 7537 2 0x00000000 2014-01-07 18:55:28 ffff881002c4f690 0000000000000046 ffff881002c4f658 ffff881002c4f654 2014-01-07 18:55:28 ffff881000000000 ffff88083fe82000 ffff880044616740 0000000000000217 2014-01-07 18:55:28 ffff881002c42638 ffff881002c4ffd8 000000000000fb88 ffff881002c42638 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] io_schedule+0x73/0xc0 2014-01-07 18:55:28 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:28 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:28 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:28 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:28 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:28 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:28 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:28 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:28 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:28 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:28 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:28 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:28 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:28 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:28 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:28 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:28 [] ? child_rip+0x0/0x20 2014-01-07 18:55:28 ll_ost_io00_0 S 0000000000000003 0 7538 2 0x00000000 2014-01-07 18:55:28 ffff880752fe7da0 0000000000000046 ffff880752fe7d68 ffff880752fe7d64 2014-01-07 18:55:28 ffff8807c0202400 ffff88083fe81600 ffff880044616740 0000000000000207 2014-01-07 18:55:28 ffff880832809098 ffff880752fe7fd8 000000000000fb88 ffff880832809098 2014-01-07 18:55:28 Call Trace: 2014-01-07 18:55:28 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:28 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:28 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:28 [] kthread+0x96/0xa0 2014-01-07 18:55:28 [] child_rip+0xa/0x20 2014-01-07 18:55:28 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io00_0 S 0000000000000000 0 7539 2 0x00000000 2014-01-07 18:55:29 ffff88075240b930 0000000000000046 0000000000000000 ffff88075240b8f4 2014-01-07 18:55:29 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:29 ffff880832809af8 ffff88075240bfd8 000000000000fb88 ffff880832809af8 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:29 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:29 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:29 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:29 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:29 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:29 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:29 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io03_0 S 000000000000000f 0 7540 2 0x00000000 2014-01-07 18:55:29 ffff881002c75da0 0000000000000046 ffff881002c75d68 ffff881002c75d64 2014-01-07 18:55:29 ffff881000000000 ffff88083fe82e00 ffff88085c496740 0000000000000207 2014-01-07 18:55:29 ffff881003161098 ffff881002c75fd8 000000000000fb88 ffff881003161098 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io02_0 S 0000000000000008 0 7541 2 0x00000000 2014-01-07 18:55:29 ffff881002c7dda0 0000000000000046 ffff881000000010 ffff881003f3d3c0 2014-01-07 18:55:29 ffff881031875c00 0000000000000000 0000000000000003 ffff8807d1a359c0 2014-01-07 18:55:29 ffff881002c77ab8 ffff881002c7dfd8 000000000000fb88 ffff881002c77ab8 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:29 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io03_0 D 000000000000000d 0 7542 2 0x00000000 2014-01-07 18:55:29 ffff881002c93690 0000000000000046 ffff881002c93658 ffff881002c93654 2014-01-07 18:55:29 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:29 ffff881002c77058 ffff881002c93fd8 000000000000fb88 ffff881002c77058 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] io_schedule+0x73/0xc0 2014-01-07 18:55:29 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:29 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:29 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:29 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:29 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:29 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:29 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:29 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:29 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:29 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:29 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:29 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:29 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:29 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:29 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io00_0 S 0000000000000002 0 7543 2 0x00000000 2014-01-07 18:55:29 ffff881002c9dda0 0000000000000046 0000000000000000 ffff881002c9dd64 2014-01-07 18:55:29 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:29 ffff881002c765f8 ffff881002c9dfd8 000000000000fb88 ffff881002c765f8 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io03_0 S 000000000000000f 0 7544 2 0x00000000 2014-01-07 18:55:29 ffff881002cc1930 0000000000000046 ffff881002cc18f8 ffff881002cc18f4 2014-01-07 18:55:29 52ccbdfe00000000 ffff88083fe82e00 ffff880044616740 0000000000000217 2014-01-07 18:55:29 ffff881002c9faf8 ffff881002cc1fd8 000000000000fb88 ffff881002c9faf8 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:29 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:29 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:29 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:29 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:29 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:29 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:29 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io01_0 S 0000000000000004 0 7545 2 0x00000000 2014-01-07 18:55:29 ffff881002cc9930 0000000000000046 0000000000000000 ffff881002cc98f4 2014-01-07 18:55:29 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:29 ffff881002c9f098 ffff881002cc9fd8 000000000000fb88 ffff881002c9f098 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:29 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:29 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:29 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:29 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:29 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:29 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:29 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io01_0 S 0000000000000006 0 7546 2 0x00000000 2014-01-07 18:55:29 ffff8807508d5930 0000000000000046 0000000000000000 ffff8807508d58f4 2014-01-07 18:55:29 00052ccb00000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:29 ffff880833e97058 ffff8807508d5fd8 000000000000fb88 ffff880833e97058 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:29 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:29 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:29 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:29 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:29 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:29 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:29 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:29 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:29 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:29 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:29 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:29 [] kthread+0x96/0xa0 2014-01-07 18:55:29 [] child_rip+0xa/0x20 2014-01-07 18:55:29 [] ? kthread+0x0/0xa0 2014-01-07 18:55:29 [] ? child_rip+0x0/0x20 2014-01-07 18:55:29 ll_ost_io02_0 S 0000000000000009 0 7547 2 0x00000000 2014-01-07 18:55:29 ffff88074fef5da0 0000000000000046 0000000000000000 ffff88074fef5d64 2014-01-07 18:55:29 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:29 ffff880831252638 ffff88074fef5fd8 000000000000fb88 ffff880831252638 2014-01-07 18:55:29 Call Trace: 2014-01-07 18:55:29 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:29 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000d 0 7548 2 0x00000000 2014-01-07 18:55:30 ffff88074f543da0 0000000000000046 0000000000000010 ffff881005231300 2014-01-07 18:55:30 ffff8810065e4000 0000000000000000 0000000000000064 ffff8807d1a359c0 2014-01-07 18:55:30 ffff8808342bfab8 ffff88074f543fd8 000000000000fb88 ffff8808342bfab8 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:30 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:30 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000d 0 7549 2 0x00000000 2014-01-07 18:55:30 ffff8807348db930 0000000000000046 ffff8807348db8f8 ffff8807348db8f4 2014-01-07 18:55:30 52ccbdff00000000 ffff88083fe82a00 ffff880044616740 000000000000020f 2014-01-07 18:55:30 ffff8808342bf058 ffff8807348dbfd8 000000000000fb88 ffff8808342bf058 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:30 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:30 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:30 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:30 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:30 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:30 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:30 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:30 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000e 0 7550 2 0x00000000 2014-01-07 18:55:30 ffff8807344fbda0 0000000000000046 0000000000000000 ffff8807344fbd64 2014-01-07 18:55:30 ffff881000000000 ffff88083fe82c00 ffff88085c476740 0000000000000181 2014-01-07 18:55:30 ffff8808323d1af8 ffff8807344fbfd8 000000000000fb88 ffff8808323d1af8 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000e 0 7551 2 0x00000000 2014-01-07 18:55:30 ffff8807344fdda0 0000000000000046 ffff8807344fdd68 ffff8807344fdd64 2014-01-07 18:55:30 ffff881000000000 ffff88083fe82c00 ffff88085c496740 00000000000002af 2014-01-07 18:55:30 ffff880833cc65f8 ffff8807344fdfd8 000000000000fb88 ffff880833cc65f8 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000d 0 7552 2 0x00000000 2014-01-07 18:55:30 ffff8807344ffda0 0000000000000046 0000000000000010 ffff881004ecd140 2014-01-07 18:55:30 ffff8810065e4000 0000000000000000 0000000000000010 ffff8807d1a359c0 2014-01-07 18:55:30 ffff880833502638 ffff8807344fffd8 000000000000fb88 ffff880833502638 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:30 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:30 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 D 000000000000000c 0 7553 2 0x00000000 2014-01-07 18:55:30 ffff880734503690 0000000000000046 ffff8807345036b0 ffffffffa04d9367 2014-01-07 18:55:30 ffff8807345036b0 0000000000000001 0000000000000db4 ffff880e00000000 2014-01-07 18:55:30 ffff880833cd1af8 ffff880734503fd8 000000000000fb88 ffff880833cd1af8 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:30 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:30 [] io_schedule+0x73/0xc0 2014-01-07 18:55:30 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:30 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:30 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:30 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:30 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:30 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:30 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:30 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:30 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:30 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:30 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:30 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:30 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:30 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:30 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 D 000000000000000e 0 7554 2 0x00000000 2014-01-07 18:55:30 ffff880733557690 0000000000000046 ffff8807335576b0 ffffffffa04d9367 2014-01-07 18:55:30 ffff8807335576b0 0000000000000001 0000000000000d79 ffff880e00000000 2014-01-07 18:55:30 ffff880833cd1098 ffff880733557fd8 000000000000fb88 ffff880833cd1098 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] ? dmu_zfetch+0x357/0xd70 [zfs] 2014-01-07 18:55:30 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:30 [] io_schedule+0x73/0xc0 2014-01-07 18:55:30 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:30 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:30 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:30 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:30 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:30 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:30 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:30 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:30 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:30 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:30 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:30 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:30 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:30 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:30 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:30 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:30 ll_ost_io03_0 S 000000000000000c 0 7555 2 0x00000000 2014-01-07 18:55:30 ffff880733161930 0000000000000046 ffff8807331618f8 ffff8807331618f4 2014-01-07 18:55:30 52ccbe0b00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:30 ffff880828e4d098 ffff880733161fd8 000000000000fb88 ffff880828e4d098 2014-01-07 18:55:30 Call Trace: 2014-01-07 18:55:30 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:30 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:30 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:30 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:30 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:30 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:30 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:30 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:30 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:30 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:30 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:30 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:30 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:30 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:30 [] kthread+0x96/0xa0 2014-01-07 18:55:30 [] child_rip+0xa/0x20 2014-01-07 18:55:30 [] ? kthread+0x0/0xa0 2014-01-07 18:55:30 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io00_0 S 0000000000000003 0 7556 2 0x00000000 2014-01-07 18:55:31 ffff88073297f930 0000000000000046 0000000000000000 ffff88073297f8f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff880831d7c5f8 ffff88073297ffd8 000000000000fb88 ffff880831d7c5f8 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io01_0 S 0000000000000005 0 7557 2 0x00000000 2014-01-07 18:55:31 ffff8807ba03d930 0000000000000046 0000000000000000 ffff8807ba03d8f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff88080dc27098 ffff8807ba03dfd8 000000000000fb88 ffff88080dc27098 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io00_0 S 0000000000000000 0 7558 2 0x00000000 2014-01-07 18:55:31 ffff8807ba03fda0 0000000000000046 0000000000000000 ffff8807ba03fd64 2014-01-07 18:55:31 ffff880700000000 ffff88083fe81000 ffff88085c4b6740 0000000000000400 2014-01-07 18:55:31 ffff8808319b3ab8 ffff8807ba03ffd8 000000000000fb88 ffff8808319b3ab8 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io00_0 S 0000000000000002 0 7559 2 0x00000000 2014-01-07 18:55:31 ffff88073299b930 0000000000000046 0000000000000000 ffff88073299b8f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff8808319a6638 ffff88073299bfd8 000000000000fb88 ffff8808319a6638 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io01_0 S 0000000000000005 0 7560 2 0x00000000 2014-01-07 18:55:31 ffff8807321cf930 0000000000000046 0000000000000000 ffff8807321cf8f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81a00 ffff880044676740 0000000000000207 2014-01-07 18:55:31 ffff8808325885f8 ffff8807321cffd8 000000000000fb88 ffff8808325885f8 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io00_0 S 0000000000000002 0 7561 2 0x00000000 2014-01-07 18:55:31 ffff880731de9930 0000000000000046 0000000000000000 ffff880731de98f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff880827287ab8 ffff880731de9fd8 000000000000fb88 ffff880827287ab8 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io00_0 S 0000000000000002 0 7562 2 0x00000000 2014-01-07 18:55:31 ffff880731e25da0 0000000000000046 0000000000000000 ffff880731e25d64 2014-01-07 18:55:31 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff880827287058 ffff880731e25fd8 000000000000fb88 ffff880827287058 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io01_0 S 0000000000000005 0 7563 2 0x00000000 2014-01-07 18:55:31 ffff880731a3b930 0000000000000046 0000000000000000 ffff880731a3b8f4 2014-01-07 18:55:31 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff88083113d058 ffff880731a3bfd8 000000000000fb88 ffff88083113d058 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:31 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:31 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:31 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:31 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:31 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:31 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:31 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:31 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:31 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:31 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:31 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:31 [] kthread+0x96/0xa0 2014-01-07 18:55:31 [] child_rip+0xa/0x20 2014-01-07 18:55:31 [] ? kthread+0x0/0xa0 2014-01-07 18:55:31 [] ? child_rip+0x0/0x20 2014-01-07 18:55:31 ll_ost_io01_0 S 0000000000000005 0 7564 2 0x00000000 2014-01-07 18:55:31 ffff880731a51da0 0000000000000046 0000000000000000 ffff880731a51d64 2014-01-07 18:55:31 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:31 ffff880828e4c638 ffff880731a51fd8 000000000000fb88 ffff880828e4c638 2014-01-07 18:55:31 Call Trace: 2014-01-07 18:55:31 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:31 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io00_0 S 0000000000000000 0 7565 2 0x00000000 2014-01-07 18:55:32 ffff880731a59930 0000000000000046 ffff880731a598f8 ffff880731a598f4 2014-01-07 18:55:32 00052ccb00000000 0000000000000282 ffff880731a598d0 ffffffff810810cc 2014-01-07 18:55:32 ffff88080dc27af8 ffff880731a59fd8 000000000000fb88 ffff88080dc27af8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:32 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:32 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:32 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:32 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:32 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:32 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io00_0 S 0000000000000000 0 7566 2 0x00000000 2014-01-07 18:55:32 ffff880731a5b930 0000000000000046 0000000000000000 ffff880731a5b8f4 2014-01-07 18:55:32 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:32 ffff880831daa638 ffff880731a5bfd8 000000000000fb88 ffff880831daa638 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:32 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:32 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:32 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:32 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:32 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:32 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io02_0 S 0000000000000009 0 7567 2 0x00000000 2014-01-07 18:55:32 ffff880731697da0 0000000000000046 0000000000000000 ffff880731697d64 2014-01-07 18:55:32 ffff881000000000 ffff88083fe82200 ffff880044636740 00000000000001ff 2014-01-07 18:55:32 ffff880833c3d098 ffff880731697fd8 000000000000fb88 ffff880833c3d098 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io01_0 S 0000000000000005 0 7568 2 0x00000000 2014-01-07 18:55:32 ffff8807312bdda0 0000000000000046 0000000000000000 ffff8807312bdd64 2014-01-07 18:55:32 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:32 ffff880833c3daf8 ffff8807312bdfd8 000000000000fb88 ffff880833c3daf8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io00_0 S 0000000000000002 0 7569 2 0x00000000 2014-01-07 18:55:32 ffff8807312c1930 0000000000000046 0000000000000000 ffff8807312c18f4 2014-01-07 18:55:32 00052ccb00000000 0000000000000286 ffff8807312c18d0 ffffffff810810cc 2014-01-07 18:55:32 ffff880827263af8 ffff8807312c1fd8 000000000000fb88 ffff880827263af8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:32 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:32 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:32 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:32 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:32 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:32 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io03_0 S 000000000000000f 0 7570 2 0x00000000 2014-01-07 18:55:32 ffff880730c4d930 0000000000000046 ffff880730c4d8f8 ffff880730c4d8f4 2014-01-07 18:55:32 52ccbe0900000000 ffff88083fe82e00 ffff880044616740 0000000000000217 2014-01-07 18:55:32 ffff8808283185f8 ffff880730c4dfd8 000000000000fb88 ffff8808283185f8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:32 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:32 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:32 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:32 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:32 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:32 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io00_0 S 0000000000000003 0 7571 2 0x00000000 2014-01-07 18:55:32 ffff880730f0d930 0000000000000046 0000000000000000 ffff880730f0d8f4 2014-01-07 18:55:32 00052ccb00000000 0000000000000286 ffff880730f0d8d0 ffffffff810810cc 2014-01-07 18:55:32 ffff880831da5ab8 ffff880730f0dfd8 000000000000fb88 ffff880831da5ab8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:32 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:32 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:32 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:32 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:32 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:32 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io00_0 S 0000000000000002 0 7572 2 0x00000000 2014-01-07 18:55:32 ffff880730369da0 0000000000000046 0000000000000000 ffff880730369d64 2014-01-07 18:55:32 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:32 ffff880831d6d098 ffff880730369fd8 000000000000fb88 ffff880831d6d098 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:32 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io03_0 S 000000000000000c 0 7573 2 0x00000000 2014-01-07 18:55:32 0000000000000000 0000000000000000 0000000000000000 ffff88100329b6c0 2014-01-07 18:55:32 0000000000000001 0000000000000000 ffff880044616740 0000000000000217 2014-01-07 18:55:32 ffff8808319bdaf8 ffff88073038bfd8 000000000000fb88 ffff8808319bdaf8 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:32 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:32 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:32 [] kthread+0x96/0xa0 2014-01-07 18:55:32 [] child_rip+0xa/0x20 2014-01-07 18:55:32 [] ? kthread+0x0/0xa0 2014-01-07 18:55:32 [] ? child_rip+0x0/0x20 2014-01-07 18:55:32 ll_ost_io02_0 S 0000000000000008 0 7574 2 0x00000000 2014-01-07 18:55:32 ffff88072fd93930 0000000000000046 ffff88072fd938f8 ffff88072fd938f4 2014-01-07 18:55:32 00052ccb00000000 ffff88083fe82000 ffff880044616740 0000000000000217 2014-01-07 18:55:32 ffff880831d1c638 ffff88072fd93fd8 000000000000fb88 ffff880831d1c638 2014-01-07 18:55:32 Call Trace: 2014-01-07 18:55:32 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:32 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:33 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:33 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:33 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:33 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:33 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:33 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:33 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:33 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:33 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:33 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io03_0 D 000000000000000d 0 7575 2 0x00000000 2014-01-07 18:55:33 ffff881002e6f690 0000000000000046 ffff881002e6f658 ffff881002e6f654 2014-01-07 18:55:33 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:33 ffff881003160638 ffff881002e6ffd8 000000000000fb88 ffff881003160638 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] io_schedule+0x73/0xc0 2014-01-07 18:55:33 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:33 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:33 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:33 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:33 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:33 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:33 [] ? dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:33 [] ? osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:33 [] ? pointer+0x8d/0x830 2014-01-07 18:55:33 [] ? pointer+0x8d/0x830 2014-01-07 18:55:33 [] ? put_dec+0x10c/0x110 2014-01-07 18:55:33 [] ? number+0x2ee/0x320 2014-01-07 18:55:33 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:33 [] ? put_dec+0x10c/0x110 2014-01-07 18:55:33 [] ? number+0x2ee/0x320 2014-01-07 18:55:33 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:33 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:33 [] ? read_tsc+0x9/0x20 2014-01-07 18:55:33 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:33 [] ? cpumask_next_and+0x29/0x50 2014-01-07 18:55:33 [] ? find_busiest_group+0x244/0x9f0 2014-01-07 18:55:33 [] ? perf_event_task_sched_out+0x33/0x80 2014-01-07 18:55:33 [] ? ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ? ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? kthread+0x96/0xa0 2014-01-07 18:55:33 [] ? child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io01_0 S 0000000000000004 0 7576 2 0x00000000 2014-01-07 18:55:33 ffff881002ee1930 0000000000000046 0000000000000000 ffff881002ee18f4 2014-01-07 18:55:33 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:33 ffff881002ebfab8 ffff881002ee1fd8 000000000000fb88 ffff881002ebfab8 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:33 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:33 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:33 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:33 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:33 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:33 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:33 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:33 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:33 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:33 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_server_post_idle_rqbds+0x38/0xe0 [ptlrpc] 2014-01-07 18:55:33 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io03_0 S 000000000000000e 0 7577 2 0x00000000 2014-01-07 18:55:33 ffff881002eeb930 0000000000000046 0000000000000000 ffff881002eeb8f4 2014-01-07 18:55:33 52ccbdfa00000000 0000000000000286 ffff881002eeb8d0 ffffffff810810cc 2014-01-07 18:55:33 ffff881002ebf058 ffff881002eebfd8 000000000000fb88 ffff881002ebf058 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:33 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:33 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:33 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:33 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:33 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:33 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:33 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:33 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:33 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:33 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:33 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io02_0 S 0000000000000008 0 7578 2 0x00000000 2014-01-07 18:55:33 ffff8810044e1da0 0000000000000046 ffff881000000010 ffff88100522ac80 2014-01-07 18:55:33 ffff881031875c00 0000000000000000 0000000000000023 ffff8807d1a359c0 2014-01-07 18:55:33 ffff8810051dfaf8 ffff8810044e1fd8 000000000000fb88 ffff8810051dfaf8 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:33 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io02_0 S 0000000000000008 0 7579 2 0x00000000 2014-01-07 18:55:33 ffff881002cf1da0 0000000000000046 0000000000000000 ffff881003ba4b40 2014-01-07 18:55:33 ffff881031875c00 0000000000000000 0000000000000026 ffff8807d1a359c0 2014-01-07 18:55:33 ffff8810051df098 ffff881002cf1fd8 000000000000fb88 ffff8810051df098 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io02_0 S 0000000000000009 0 7580 2 0x00000000 2014-01-07 18:55:33 ffff881002fdbda0 0000000000000046 0000000000000000 ffff881002fdbd64 2014-01-07 18:55:33 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:33 ffff881002fd9ab8 ffff881002fdbfd8 000000000000fb88 ffff881002fd9ab8 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost_io02_0 S 0000000000000008 0 7581 2 0x00000000 2014-01-07 18:55:33 ffff8810033cfda0 0000000000000046 ffff8810033cfd68 ffff8810033cfd64 2014-01-07 18:55:33 ffff881031875c00 ffff88083fe82000 ffff88085c436740 0000000000000400 2014-01-07 18:55:33 ffff881002ebe5f8 ffff8810033cffd8 000000000000fb88 ffff881002ebe5f8 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost01_005 S 0000000000000007 0 7984 2 0x00000000 2014-01-07 18:55:33 ffff880e90001da0 0000000000000046 0000000000000000 ffff8807d2d9a2c0 2014-01-07 18:55:33 ffff8807d16ff000 0000000000000000 000000000000004d ffff8807d1a36080 2014-01-07 18:55:33 ffff881007272638 ffff880e90001fd8 000000000000fb88 ffff881007272638 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:33 [] ? child_rip+0x0/0x20 2014-01-07 18:55:33 ll_ost01_006 S 0000000000000006 0 7985 2 0x00000000 2014-01-07 18:55:33 ffff880e8ca1dda0 0000000000000046 ffff880e8ca1dd68 ffff880e8ca1dd64 2014-01-07 18:55:33 ffff8807d16ff000 ffff88083fe81c00 ffff8800446b6740 0000000000000400 2014-01-07 18:55:33 ffff881002fd85f8 ffff880e8ca1dfd8 000000000000fb88 ffff881002fd85f8 2014-01-07 18:55:33 Call Trace: 2014-01-07 18:55:33 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:33 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:33 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:33 [] kthread+0x96/0xa0 2014-01-07 18:55:33 [] child_rip+0xa/0x20 2014-01-07 18:55:33 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_007 S 0000000000000006 0 7986 2 0x00000000 2014-01-07 18:55:34 ffff880e8fc45da0 0000000000000046 ffff880e8fc45d68 ffff880e8fc45d64 2014-01-07 18:55:34 ffff8807d16ff000 ffff88083fe81c00 ffff8800446b6740 0000000000000400 2014-01-07 18:55:34 ffff88082884f058 ffff880e8fc45fd8 000000000000fb88 ffff88082884f058 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_008 S 0000000000000006 0 7987 2 0x00000000 2014-01-07 18:55:34 ffff880e903a7da0 0000000000000046 0000000000000000 ffff880e903a7d64 2014-01-07 18:55:34 ffff880700000000 ffff88083fe81c00 ffff880044636740 000000000000015a 2014-01-07 18:55:34 ffff8808343ee5f8 ffff880e903a7fd8 000000000000fb88 ffff8808343ee5f8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_009 S 0000000000000005 0 7988 2 0x00000000 2014-01-07 18:55:34 ffff880e90115da0 0000000000000046 ffff880e90115d68 ffff880e90115d64 2014-01-07 18:55:34 ffff8807d16ff000 ffff88083fe81e00 ffff880044696740 0000000000000400 2014-01-07 18:55:34 ffff88083191c5f8 ffff880e90115fd8 000000000000fb88 ffff88083191c5f8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_010 S 0000000000000004 0 7989 2 0x00000000 2014-01-07 18:55:34 ffff880e8fc65da0 0000000000000046 0000000000000010 ffff8807b4603380 2014-01-07 18:55:34 ffff8807d16ff000 0000000000000000 000000000000000c ffff8807d1a36080 2014-01-07 18:55:34 ffff8808271b3098 ffff880e8fc65fd8 000000000000fb88 ffff8808271b3098 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_011 S 0000000000000006 0 7990 2 0x00000000 2014-01-07 18:55:34 ffff880e8fc27da0 0000000000000046 0000000000000000 ffff88007f877740 2014-01-07 18:55:34 ffff8807d16ff000 0000000000000000 0000000000000004 ffff8807d1a36080 2014-01-07 18:55:34 ffff88082833d098 ffff880e8fc27fd8 000000000000fb88 ffff88082833d098 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost01_012 S 0000000000000004 0 7991 2 0x00000000 2014-01-07 18:55:34 ffff880e8bc47da0 0000000000000046 ffff880e8bc47d68 ffff880e8bc47d64 2014-01-07 18:55:34 ffff8807d16ff000 ffff88083fe81800 ffff8800446f6740 0000000000000400 2014-01-07 18:55:34 ffff88082833daf8 ffff880e8bc47fd8 000000000000fb88 ffff88082833daf8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ldlm_cn03_003 S 000000000000000c 0 7992 2 0x00000000 2014-01-07 18:55:34 ffff880e8ca0fda0 0000000000000046 ffff880e8ca0fd68 ffff880e8ca0fd64 2014-01-07 18:55:34 ffff881030b29800 ffff88083fe82800 ffff88085c4d6740 0000000000000400 2014-01-07 18:55:34 ffff881002fd9058 ffff880e8ca0ffd8 000000000000fb88 ffff881002fd9058 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost00_005 S 0000000000000002 0 7993 2 0x00000000 2014-01-07 18:55:34 ffff880e8c001da0 0000000000000046 0000000000000000 ffff880e8c001d64 2014-01-07 18:55:34 ffff880700000000 ffff88083fe81200 ffff88085c4f6740 0000000000000207 2014-01-07 18:55:34 ffff8810033e25f8 ffff880e8c001fd8 000000000000fb88 ffff8810033e25f8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ldlm_cn02_003 S 0000000000000008 0 7994 2 0x00000000 2014-01-07 18:55:34 ffff880e8d9cbda0 0000000000000046 0000000000000000 ffff88100398e8c0 2014-01-07 18:55:34 ffff881030b29400 0000000000000000 0000000000000046 ffff8807d1a35dc0 2014-01-07 18:55:34 ffff880e903f3af8 ffff880e8d9cbfd8 000000000000fb88 ffff880e903f3af8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost_io01_0 S 0000000000000007 0 7995 2 0x00000000 2014-01-07 18:55:34 ffff880e8b0d7930 0000000000000046 0000000000000000 ffff880e8b0d78f4 2014-01-07 18:55:34 00052ccb00000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:34 ffff880e903f3098 ffff880e8b0d7fd8 000000000000fb88 ffff880e903f3098 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:34 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:34 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:34 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:34 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:34 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:34 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:34 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:34 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:34 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:34 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_server_post_idle_rqbds+0x38/0xe0 [ptlrpc] 2014-01-07 18:55:34 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost_io01_0 S 0000000000000005 0 7996 2 0x00000000 2014-01-07 18:55:34 ffff880e8d633da0 0000000000000046 0000000000000000 ffff880e8d633d64 2014-01-07 18:55:34 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:34 ffff880e903f2638 ffff880e8d633fd8 000000000000fb88 ffff880e903f2638 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:34 [] kthread+0x96/0xa0 2014-01-07 18:55:34 [] child_rip+0xa/0x20 2014-01-07 18:55:34 [] ? kthread+0x0/0xa0 2014-01-07 18:55:34 [] ? child_rip+0x0/0x20 2014-01-07 18:55:34 ll_ost_io02_0 S 0000000000000009 0 7997 2 0x00000000 2014-01-07 18:55:34 ffff880e8d747930 0000000000000046 ffff880e8d7478f8 ffff880e8d7478f4 2014-01-07 18:55:34 00052ccb00000000 0000000000000286 ffff880e8d7478d0 ffffffff810810cc 2014-01-07 18:55:34 ffff880e8da49ab8 ffff880e8d747fd8 000000000000fb88 ffff880e8da49ab8 2014-01-07 18:55:34 Call Trace: 2014-01-07 18:55:34 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:34 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:34 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:34 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:34 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:34 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:34 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:34 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:34 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:34 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:34 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:34 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:34 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:34 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io00_0 S 0000000000000003 0 7998 2 0x00000000 2014-01-07 18:55:35 ffff880e867e7930 0000000000000046 0000000000000000 ffff880e867e78f4 2014-01-07 18:55:35 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:35 ffff880e8da49058 ffff880e867e7fd8 000000000000fb88 ffff880e8da49058 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:35 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:35 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:35 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:35 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io00_0 S 0000000000000000 0 7999 2 0x00000000 2014-01-07 18:55:35 ffff880e867e9930 0000000000000046 ffff880e867e98f8 ffff880e867e98f4 2014-01-07 18:55:35 ffff880e00000000 0000000000000282 ffff880e867e98d0 ffffffff810810cc 2014-01-07 18:55:35 ffff880e8da485f8 ffff880e867e9fd8 000000000000fb88 ffff880e8da485f8 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:35 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:35 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:35 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:35 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:35 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ldlm_cn00_003 S 0000000000000001 0 8000 2 0x00000000 2014-01-07 18:55:35 ffff880e85d07da0 0000000000000046 0000000000000000 ffff880e85d07d64 2014-01-07 18:55:35 ffff8807d16c0000 ffff88083fe81000 ffff880044676740 0000000000000400 2014-01-07 18:55:35 ffff880e85d05af8 ffff880e85d07fd8 000000000000fb88 ffff880e85d05af8 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io02_0 S 000000000000000b 0 8001 2 0x00000000 2014-01-07 18:55:35 ffff880e85de5da0 0000000000000046 0000000000000000 ffff881002b2ed80 2014-01-07 18:55:35 ffff881031875c00 0000000000000000 0000000000000062 ffff8807d1a359c0 2014-01-07 18:55:35 ffff880e85de3ab8 ffff880e85de5fd8 000000000000fb88 ffff880e85de3ab8 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io00_0 S 0000000000000003 0 8002 2 0x00000000 2014-01-07 18:55:35 ffff880e85e87930 0000000000000046 0000000000000000 ffff880e85e878f4 2014-01-07 18:55:35 00052ccb00000000 0000000000000286 ffff880e85e878d0 ffffffff810810cc 2014-01-07 18:55:35 ffff880e85de3058 ffff880e85e87fd8 000000000000fb88 ffff880e85de3058 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:35 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:35 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:35 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:35 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:35 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io01_0 D 0000000000000005 0 8003 2 0x00000000 2014-01-07 18:55:35 ffff880e85f05690 0000000000000046 0000000000000000 ffff880e85f05654 2014-01-07 18:55:35 ffff880e00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:35 ffff880e85de25f8 ffff880e85f05fd8 000000000000fb88 ffff880e85de25f8 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] io_schedule+0x73/0xc0 2014-01-07 18:55:35 [] cv_wait_common+0x8c/0x100 [spl] 2014-01-07 18:55:35 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:35 [] __cv_wait_io+0x18/0x20 [spl] 2014-01-07 18:55:35 [] zio_wait+0xfb/0x1b0 [zfs] 2014-01-07 18:55:35 [] dmu_buf_hold_array_by_dnode+0x1dd/0x560 [zfs] 2014-01-07 18:55:35 [] dmu_buf_hold_array_by_bonus+0x68/0x90 [zfs] 2014-01-07 18:55:35 [] osd_bufs_get+0x493/0xb00 [osd_zfs] 2014-01-07 18:55:35 [] ? libcfs_log_return+0x28/0x40 [libcfs] 2014-01-07 18:55:35 [] ofd_preprw_read+0x14b/0x7f0 [ofd] 2014-01-07 18:55:35 [] ofd_preprw+0x77a/0x1480 [ofd] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] obd_preprw+0x121/0x390 [ost] 2014-01-07 18:55:35 [] ost_brw_read+0xd29/0x1350 [ost] 2014-01-07 18:55:35 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io03_0 S 000000000000000d 0 8004 2 0x00000000 2014-01-07 18:55:35 ffff880e85f6b930 0000000000000046 ffff880e85f6b8f8 ffff880e85f6b8f4 2014-01-07 18:55:35 52ccbe0100000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:35 ffff880e85f69af8 ffff880e85f6bfd8 000000000000fb88 ffff880e85f69af8 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:35 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:35 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:35 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:35 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:35 ll_ost_io01_0 S 0000000000000004 0 8005 2 0x00000000 2014-01-07 18:55:35 ffff880e85f79930 0000000000000046 0000000000000000 ffff880e85f798f4 2014-01-07 18:55:35 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:35 ffff880e85f69098 ffff880e85f79fd8 000000000000fb88 ffff880e85f69098 2014-01-07 18:55:35 Call Trace: 2014-01-07 18:55:35 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:35 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:35 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:35 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:35 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:35 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:35 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:35 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:35 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:35 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:35 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:35 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:35 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:35 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:35 [] kthread+0x96/0xa0 2014-01-07 18:55:35 [] child_rip+0xa/0x20 2014-01-07 18:55:35 [] ? kthread+0x0/0xa0 2014-01-07 18:55:35 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io00_0 S 0000000000000003 0 8006 2 0x00000000 2014-01-07 18:55:36 ffff880e85fa9da0 0000000000000046 0000000000000000 ffff880e85fa9d64 2014-01-07 18:55:36 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:36 ffff880e85f68638 ffff880e85fa9fd8 000000000000fb88 ffff880e85f68638 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io01_0 S 0000000000000005 0 8007 2 0x00000000 2014-01-07 18:55:36 ffff880e85fcbda0 0000000000000046 ffff880e85fcbd68 ffff880e85fcbd64 2014-01-07 18:55:36 ffff8807c0202000 ffff88083fe81a00 ffff880044616740 0000000000000209 2014-01-07 18:55:36 ffff8810051de638 ffff880e85fcbfd8 000000000000fb88 ffff8810051de638 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io00_0 S 0000000000000003 0 8008 2 0x00000000 2014-01-07 18:55:36 ffff880e85fcd930 0000000000000046 ffff880e85fcd8f8 ffff880e85fcd8f4 2014-01-07 18:55:36 00052ccb00000000 ffff88083fe81600 ffff880044656740 0000000000000400 2014-01-07 18:55:36 ffff88100725a638 ffff880e85fcdfd8 000000000000fb88 ffff88100725a638 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:36 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:36 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:36 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:36 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:36 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:36 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:36 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:36 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:36 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:36 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io01_0 S 0000000000000004 0 8009 2 0x00000000 2014-01-07 18:55:36 ffff880e8582d930 0000000000000046 0000000000000000 ffff880e8582d8f4 2014-01-07 18:55:36 00052ccb00000000 0000000000000286 ffff880e8582d8d0 ffffffff810810cc 2014-01-07 18:55:36 ffff880e8582bab8 ffff880e8582dfd8 000000000000fb88 ffff880e8582bab8 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:36 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:36 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:36 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:36 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:36 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:36 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:36 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:36 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:36 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:36 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:36 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io00_0 S 0000000000000002 0 8050 2 0x00000000 2014-01-07 18:55:36 ffff880e84e33930 0000000000000046 0000000000000000 ffff880e84e338f4 2014-01-07 18:55:36 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:36 ffff880e8582b058 ffff880e84e33fd8 000000000000fb88 ffff880e8582b058 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:36 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:36 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:36 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:36 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:36 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:36 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:36 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:36 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:36 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:36 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io02_0 S 0000000000000008 0 8051 2 0x00000000 2014-01-07 18:55:36 ffff880e8463bda0 0000000000000046 0000000000000000 ffff880e8463bd64 2014-01-07 18:55:36 ffff881000000000 ffff88083fe82000 ffff88085c476740 000000000000015a 2014-01-07 18:55:36 ffff88100723b058 ffff880e8463bfd8 000000000000fb88 ffff88100723b058 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io02_0 S 0000000000000008 0 8052 2 0x00000000 2014-01-07 18:55:36 ffff880e847f9da0 0000000000000046 0000000000000000 ffff880e84523800 2014-01-07 18:55:36 ffff881031875c00 0000000000000000 0000000000000003 ffff8807d1a359c0 2014-01-07 18:55:36 ffff880e85183af8 ffff880e847f9fd8 000000000000fb88 ffff880e85183af8 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io02_0 S 000000000000000a 0 8053 2 0x00000000 2014-01-07 18:55:36 ffff880e84bd1da0 0000000000000046 ffff881000000010 ffff880e8521ac00 2014-01-07 18:55:36 ffff881031875c00 0000000000000000 0000000000000046 ffff8807d1a359c0 2014-01-07 18:55:36 ffff880e847c3058 ffff880e84bd1fd8 000000000000fb88 ffff880e847c3058 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:36 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io01_0 S 0000000000000005 0 8054 2 0x00000000 2014-01-07 18:55:36 ffff880e84463da0 0000000000000046 0000000000000000 ffff880e84463d64 2014-01-07 18:55:36 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:36 ffff880e847c3ab8 ffff880e84463fd8 000000000000fb88 ffff880e847c3ab8 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io03_0 S 000000000000000d 0 8055 2 0x00000000 2014-01-07 18:55:36 ffff880e83d0b930 0000000000000046 ffff880e83d0b8f8 ffff880e83d0b8f4 2014-01-07 18:55:36 52ccbe0100000000 0000000000000286 ffff880e83d0b8d0 ffffffff810810cc 2014-01-07 18:55:36 ffff880e847d0638 ffff880e83d0bfd8 000000000000fb88 ffff880e847d0638 2014-01-07 18:55:36 Call Trace: 2014-01-07 18:55:36 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:36 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:36 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:36 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:36 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:36 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:36 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:36 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:36 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:36 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:36 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:36 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:36 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:36 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:36 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:36 [] kthread+0x96/0xa0 2014-01-07 18:55:36 [] child_rip+0xa/0x20 2014-01-07 18:55:36 [] ? kthread+0x0/0xa0 2014-01-07 18:55:36 [] ? child_rip+0x0/0x20 2014-01-07 18:55:36 ll_ost_io01_0 S 0000000000000004 0 8056 2 0x00000000 2014-01-07 18:55:36 ffff880e84619da0 0000000000000046 0000000000000000 ffff880e84619d64 2014-01-07 18:55:37 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:37 ffff880e847d1098 ffff880e84619fd8 000000000000fb88 ffff880e847d1098 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io00_0 S 0000000000000002 0 8057 2 0x00000000 2014-01-07 18:55:37 ffff880e84f21930 0000000000000046 ffff880e84f218f8 ffff880e84f218f4 2014-01-07 18:55:37 00052ccb00000000 0000000000000286 ffff880e84f218d0 ffffffff810810cc 2014-01-07 18:55:37 ffff880e847c25f8 ffff880e84f21fd8 000000000000fb88 ffff880e847c25f8 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:37 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:37 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:37 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:37 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:37 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:37 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:37 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:37 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:37 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:37 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:37 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io01_0 S 0000000000000007 0 8058 2 0x00000000 2014-01-07 18:55:37 ffff880e845cd930 0000000000000046 0000000000000000 ffff880e845cd8f4 2014-01-07 18:55:37 00052ccb00000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:37 ffff880e84bec638 ffff880e845cdfd8 000000000000fb88 ffff880e84bec638 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:37 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:37 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:37 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:37 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:37 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:37 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:37 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:37 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:37 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:37 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ldlm_cn01_003 S 0000000000000007 0 8059 2 0x00000000 2014-01-07 18:55:37 ffff880e84333da0 0000000000000046 ffff880e84333d68 ffff880e84333d64 2014-01-07 18:55:37 ffff880700000000 ffff88083fe81e00 ffff880044696740 0000000000000503 2014-01-07 18:55:37 ffff880e84bed098 ffff880e84333fd8 000000000000fb88 ffff880e84bed098 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 snmpd S 000000000000000a 0 8612 1 0x00000000 2014-01-07 18:55:37 ffff880e8494d908 0000000000000082 0000000000000000 ffff880840010dc0 2014-01-07 18:55:37 ffffffff8113b549 0000000000000002 0000000000000003 ffff880840021b40 2014-01-07 18:55:37 ffff88103038a638 ffff880e8494dfd8 000000000000fb88 ffff88103038a638 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ? zone_statistics+0x99/0xc0 2014-01-07 18:55:37 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:55:37 [] ? add_wait_queue+0x46/0x60 2014-01-07 18:55:37 [] ? __pollwait+0x75/0xf0 2014-01-07 18:55:37 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:55:37 [] do_select+0x57c/0x6c0 2014-01-07 18:55:37 [] ? __pollwait+0x0/0xf0 2014-01-07 18:55:37 [] ? pollwake+0x0/0x60 2014-01-07 18:55:37 [] ? pollwake+0x0/0x60 2014-01-07 18:55:37 [] ? pollwake+0x0/0x60 2014-01-07 18:55:37 [] ? filemap_fault+0xd3/0x500 2014-01-07 18:55:37 [] ? unlock_page+0x27/0x30 2014-01-07 18:55:37 [] ? __do_fault+0x469/0x530 2014-01-07 18:55:37 [] ? handle_pte_fault+0xf7/0xb50 2014-01-07 18:55:37 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:37 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:55:37 [] ? __do_page_fault+0x1ec/0x480 2014-01-07 18:55:37 [] ? sys_sendto+0x139/0x190 2014-01-07 18:55:37 [] ? mntput_no_expire+0x30/0x110 2014-01-07 18:55:37 [] sys_select+0x47/0x110 2014-01-07 18:55:37 [] ? do_page_fault+0x3e/0xa0 2014-01-07 18:55:37 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:37 ll_ost_io02_0 S 0000000000000008 0 8895 2 0x00000000 2014-01-07 18:55:37 ffff880e8bd35930 0000000000000046 ffff880e8bd358f8 ffff880e8bd358f4 2014-01-07 18:55:37 00052ccb00000000 ffff88083fe82000 ffff880044616740 0000000000000217 2014-01-07 18:55:37 ffff880e84b32638 ffff880e8bd35fd8 000000000000fb88 ffff880e84b32638 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:37 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:37 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:37 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:37 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:37 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:37 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:37 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:37 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:37 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:37 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:37 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io01_0 S 0000000000000004 0 8896 2 0x00000000 2014-01-07 18:55:37 ffff880e84065da0 0000000000000046 0000000000000000 ffff880e84065d64 2014-01-07 18:55:37 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:37 ffff880832811af8 ffff880e84065fd8 000000000000fb88 ffff880832811af8 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io02_0 S 000000000000000a 0 8897 2 0x00000000 2014-01-07 18:55:37 ffff880e85263da0 0000000000000046 0000000000000000 ffff880e85263d64 2014-01-07 18:55:37 ffff881000000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:55:37 ffff880828e4f058 ffff880e85263fd8 000000000000fb88 ffff880828e4f058 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost01_013 S 0000000000000007 0 8898 2 0x00000000 2014-01-07 18:55:37 ffff880e84bffda0 0000000000000046 ffff880700000010 ffff8800bbd7a380 2014-01-07 18:55:37 ffff8807d16ff000 0000000000000000 0000000000000008 ffff8807d1a36080 2014-01-07 18:55:37 ffff880832605af8 ffff880e84bfffd8 000000000000fb88 ffff880832605af8 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:37 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io00_0 S 0000000000000000 0 8899 2 0x00000000 2014-01-07 18:55:37 ffff880e8504dda0 0000000000000046 0000000000000000 ffff880e8504dd64 2014-01-07 18:55:37 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:37 ffff880831e11ab8 ffff880e8504dfd8 000000000000fb88 ffff880831e11ab8 2014-01-07 18:55:37 Call Trace: 2014-01-07 18:55:37 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:37 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:37 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:37 [] kthread+0x96/0xa0 2014-01-07 18:55:37 [] child_rip+0xa/0x20 2014-01-07 18:55:37 [] ? kthread+0x0/0xa0 2014-01-07 18:55:37 [] ? child_rip+0x0/0x20 2014-01-07 18:55:37 ll_ost_io00_0 S 0000000000000002 0 8900 2 0x00000000 2014-01-07 18:55:37 ffff880e8504fda0 0000000000000046 0000000000000000 ffff880e8504fd64 2014-01-07 18:55:37 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:37 ffff880831a38638 ffff880e8504ffd8 000000000000fb88 ffff880831a38638 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io03_0 S 000000000000000c 0 8901 2 0x00000000 2014-01-07 18:55:38 ffff880e85051930 0000000000000046 ffff880e850518f8 ffff880e850518f4 2014-01-07 18:55:38 52ccbe1000000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:38 ffff88083108baf8 ffff880e85051fd8 000000000000fb88 ffff88083108baf8 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:38 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:38 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:38 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:38 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:38 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:38 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:38 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:38 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:38 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:38 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io03_0 S 000000000000000d 0 9306 2 0x00000000 2014-01-07 18:55:38 ffff880e8285f930 0000000000000046 ffff880e8285f8f8 ffff880e8285f8f4 2014-01-07 18:55:38 52ccbe0b00000000 ffff88083fe82a00 ffff880044616740 0000000000000416 2014-01-07 18:55:38 ffff88083108b098 ffff880e8285ffd8 000000000000fb88 ffff88083108b098 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:38 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:38 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:38 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:38 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:38 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:38 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:38 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:38 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:38 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:38 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 anacron R running task 0 9359 1 0x00000008 2014-01-07 18:55:38 0000000000000088 0000000000001000 0000000000000008 0000000052ccbe05 2014-01-07 18:55:38 0000000000000000 0000000052ccbe05 0000000000000000 0000000052ccbe05 2014-01-07 18:55:38 0000000019a1ed4f 0000000000000000 0000000000000000 0000000000000000 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] ? sys_newfstat+0x33/0x40 2014-01-07 18:55:38 [] ? sys_clone+0x28/0x30 2014-01-07 18:55:38 [] ? stub_clone+0x13/0x20 2014-01-07 18:55:38 [] ? system_call_fastpath+0x16/0x1b 2014-01-07 18:55:38 ldlm_cn03_004 S 000000000000000c 0 9362 2 0x00000000 2014-01-07 18:55:38 ffff880e82bd1da0 0000000000000046 ffff880e82bd1d68 ffff880e82bd1d64 2014-01-07 18:55:38 ffff881000000000 ffff88083fe82800 ffff880044636740 000000000000ad74 2014-01-07 18:55:38 ffff8808278265f8 ffff880e82bd1fd8 000000000000fb88 ffff8808278265f8 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io01_0 S 0000000000000004 0 9765 2 0x00000000 2014-01-07 18:55:38 ffff880e8245f930 0000000000000046 0000000000000000 ffff880e8245f8f4 2014-01-07 18:55:38 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:38 ffff8808271845f8 ffff880e8245ffd8 000000000000fb88 ffff8808271845f8 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:38 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:38 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:38 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:38 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:38 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:38 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:38 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:38 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:38 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:38 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io01_0 S 0000000000000004 0 9766 2 0x00000000 2014-01-07 18:55:38 ffff880e819f5930 0000000000000046 0000000000000000 ffff880e819f58f4 2014-01-07 18:55:38 00052ccb00000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:38 ffff880834340638 ffff880e819f5fd8 000000000000fb88 ffff880834340638 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:38 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:38 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:38 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:38 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:38 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:38 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:38 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:38 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:38 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:38 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io02_0 S 000000000000000a 0 9767 2 0x00000000 2014-01-07 18:55:38 ffff880e820ddda0 0000000000000046 0000000000000000 ffff880e820ddd64 2014-01-07 18:55:38 ffff881000000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:55:38 ffff880832589ab8 ffff880e820ddfd8 000000000000fb88 ffff880832589ab8 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost_io00_0 S 0000000000000000 0 9768 2 0x00000000 2014-01-07 18:55:38 ffff880e83e47930 0000000000000046 0000000000000000 ffff880e83e478f4 2014-01-07 18:55:38 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:38 ffff880831a4faf8 ffff880e83e47fd8 000000000000fb88 ffff880831a4faf8 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:38 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:38 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:38 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:38 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:38 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:38 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:38 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:38 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:38 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:38 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost02_005 S 000000000000000b 0 10300 2 0x00000000 2014-01-07 18:55:38 ffff8807d5009da0 0000000000000046 ffff8807d5009d68 ffff8807d5009d64 2014-01-07 18:55:38 ffff881030b29c00 ffff88083fe82600 ffff88085c436740 0000000000000400 2014-01-07 18:55:38 ffff880831d1d098 ffff8807d5009fd8 000000000000fb88 ffff880831d1d098 2014-01-07 18:55:38 Call Trace: 2014-01-07 18:55:38 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:38 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:38 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:38 [] kthread+0x96/0xa0 2014-01-07 18:55:38 [] child_rip+0xa/0x20 2014-01-07 18:55:38 [] ? kthread+0x0/0xa0 2014-01-07 18:55:38 [] ? child_rip+0x0/0x20 2014-01-07 18:55:38 ll_ost03_005 S 000000000000000f 0 10301 2 0x00000000 2014-01-07 18:55:38 ffff8807d500bda0 0000000000000046 ffff8807d500bd68 ffff8807d500bd64 2014-01-07 18:55:38 ffff881031875000 ffff88083fe82c00 ffff88085c4f6740 0000000000000400 2014-01-07 18:55:38 ffff880833ffbaf8 ffff8807d500bfd8 000000000000fb88 ffff880833ffbaf8 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000f 0 10422 2 0x00000000 2014-01-07 18:55:39 ffff881005f69da0 0000000000000046 0000000000000000 ffff881005f69d64 2014-01-07 18:55:39 ffff881000000000 ffff88083fe82e00 ffff88085c476740 000000000000015a 2014-01-07 18:55:39 ffff880832605098 ffff881005f69fd8 000000000000fb88 ffff880832605098 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000c 0 10423 2 0x00000000 2014-01-07 18:55:39 ffff881005f6b930 0000000000000046 ffff881005f6b8f8 ffff881005f6b8f4 2014-01-07 18:55:39 52ccbdfb00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:39 ffff880831da45f8 ffff881005f6bfd8 000000000000fb88 ffff880831da45f8 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:39 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:39 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:39 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:39 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:39 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:39 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:39 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:39 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:39 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:39 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000e 0 10424 2 0x00000000 2014-01-07 18:55:39 ffff881005e2dda0 0000000000000046 ffff881005e2dd68 ffff881005e2dd64 2014-01-07 18:55:39 ffff881000000000 ffff88083fe82c00 ffff880044616740 0000000000000416 2014-01-07 18:55:39 ffff8808327985f8 ffff881005e2dfd8 000000000000fb88 ffff8808327985f8 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000d 0 10425 2 0x00000000 2014-01-07 18:55:39 ffff881005e2f930 0000000000000046 0000000000000000 ffff881005e2f8f4 2014-01-07 18:55:39 52ccbe0d00000000 ffff88083fe82a00 ffff88085c476740 000000000000015a 2014-01-07 18:55:39 ffff880832799058 ffff881005e2ffd8 000000000000fb88 ffff880832799058 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:39 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:39 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:39 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:39 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:39 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:39 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:39 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:39 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:39 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:39 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000e 0 10426 2 0x00000000 2014-01-07 18:55:39 ffff881005961da0 0000000000000046 0000000000000000 ffff881005961d64 2014-01-07 18:55:39 ffff881000000000 ffff88083fe82c00 ffff88085c476740 000000000000015a 2014-01-07 18:55:39 ffff8808334d8638 ffff881005961fd8 000000000000fb88 ffff8808334d8638 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io00_0 S 0000000000000002 0 10427 2 0x00000000 2014-01-07 18:55:39 ffff881005963930 0000000000000046 ffff8810059638f8 ffff8810059638f4 2014-01-07 18:55:39 00052ccb00000000 ffff88083fe81400 ffff880044616740 0000000000000207 2014-01-07 18:55:39 ffff88083434dab8 ffff881005963fd8 000000000000fb88 ffff88083434dab8 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:39 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:39 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:39 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:39 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:39 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:39 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:39 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:39 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:39 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:39 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000d 0 10428 2 0x00000000 2014-01-07 18:55:39 ffff881005fc9da0 0000000000000046 ffff881005fc9d68 ffff881005fc9d64 2014-01-07 18:55:39 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000417 2014-01-07 18:55:39 ffff88083434d058 ffff881005fc9fd8 000000000000fb88 ffff88083434d058 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000e 0 10429 2 0x00000000 2014-01-07 18:55:39 ffff881005923da0 0000000000000046 0000000000000010 ffff881005287ec0 2014-01-07 18:55:39 ffff8810065e4000 0000000000000000 000000000000002a ffff8807d1a359c0 2014-01-07 18:55:39 ffff88082830e638 ffff881005923fd8 000000000000fb88 ffff88082830e638 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:39 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io03_0 S 000000000000000e 0 10430 2 0x00000000 2014-01-07 18:55:39 ffff8810058b9da0 0000000000000046 0000000000000000 ffff8810058b9d64 2014-01-07 18:55:39 ffff881000000000 ffff88083fe82c00 ffff88085c476740 0000000000000181 2014-01-07 18:55:39 ffff88083242a638 ffff8810058b9fd8 000000000000fb88 ffff88083242a638 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io00_0 S 0000000000000003 0 10431 2 0x00000000 2014-01-07 18:55:39 ffff8810058bbda0 0000000000000046 0000000000000000 ffff8810058bbd64 2014-01-07 18:55:39 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:39 ffff880833ec1058 ffff8810058bbfd8 000000000000fb88 ffff880833ec1058 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:39 [] kthread+0x96/0xa0 2014-01-07 18:55:39 [] child_rip+0xa/0x20 2014-01-07 18:55:39 [] ? kthread+0x0/0xa0 2014-01-07 18:55:39 [] ? child_rip+0x0/0x20 2014-01-07 18:55:39 ll_ost_io01_0 S 0000000000000007 0 10432 2 0x00000000 2014-01-07 18:55:39 ffff881005ef1930 0000000000000046 0000000000000000 ffff881005ef18f4 2014-01-07 18:55:39 00052ccb00000000 ffff88083fe81e00 ffff880044636740 000000000000015a 2014-01-07 18:55:39 ffff880828d93058 ffff881005ef1fd8 000000000000fb88 ffff880828d93058 2014-01-07 18:55:39 Call Trace: 2014-01-07 18:55:39 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:39 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:39 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:39 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:39 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:39 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:39 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:39 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:39 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:39 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:39 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io01_0 S 0000000000000005 0 10433 2 0x00000000 2014-01-07 18:55:40 ffff881005ef3930 0000000000000046 0000000000000000 ffff881005ef38f4 2014-01-07 18:55:40 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:40 ffff88083191dab8 ffff881005ef3fd8 000000000000fb88 ffff88083191dab8 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io00_0 S 0000000000000003 0 10434 2 0x00000000 2014-01-07 18:55:40 ffff881005ef5930 0000000000000046 0000000000000000 ffff881005ef58f4 2014-01-07 18:55:40 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:40 ffff8808334ed098 ffff881005ef5fd8 000000000000fb88 ffff8808334ed098 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io01_0 S 0000000000000004 0 10435 2 0x00000000 2014-01-07 18:55:40 ffff881005ef7da0 0000000000000046 0000000000000000 ffff881005ef7d64 2014-01-07 18:55:40 ffff880700000000 ffff88083fe81800 ffff880044636740 000000000000015a 2014-01-07 18:55:40 ffff8808334edaf8 ffff881005ef7fd8 000000000000fb88 ffff8808334edaf8 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io01_0 S 0000000000000005 0 10436 2 0x00000000 2014-01-07 18:55:40 ffff881005ee9da0 0000000000000046 0000000000000000 ffff881005ee9d64 2014-01-07 18:55:40 ffff880700000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:40 ffff880832f225f8 ffff881005ee9fd8 000000000000fb88 ffff880832f225f8 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io00_0 S 0000000000000000 0 10437 2 0x00000000 2014-01-07 18:55:40 ffff881005eeb930 0000000000000046 0000000000000000 ffff881005eeb8f4 2014-01-07 18:55:40 00052ccb00000000 ffff88083fe81000 ffff880044636740 0000000000000181 2014-01-07 18:55:40 ffff880832f23058 ffff881005eebfd8 000000000000fb88 ffff880832f23058 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io01_0 S 0000000000000006 0 10438 2 0x00000000 2014-01-07 18:55:40 ffff881005eed930 0000000000000046 0000000000000000 ffff881005eed8f4 2014-01-07 18:55:40 00052ccb00000000 0000000000000286 ffff881005eed8d0 ffffffff810810cc 2014-01-07 18:55:40 ffff880831a445f8 ffff881005eedfd8 000000000000fb88 ffff880831a445f8 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io00_0 S 0000000000000002 0 10439 2 0x00000000 2014-01-07 18:55:40 ffff881005eef930 0000000000000046 0000000000000000 ffff881005eef8f4 2014-01-07 18:55:40 0000000000000000 0000000000000286 ffff881005eef8d0 ffffffff810810cc 2014-01-07 18:55:40 ffff8800bc659ab8 ffff881005eeffd8 000000000000fb88 ffff8800bc659ab8 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:40 [] ? kthread+0x0/0xa0 2014-01-07 18:55:40 [] ? child_rip+0x0/0x20 2014-01-07 18:55:40 ll_ost_io01_0 S 0000000000000005 0 10440 2 0x00000000 2014-01-07 18:55:40 ffff881005f91930 0000000000000046 0000000000000000 ffff881005f918f4 2014-01-07 18:55:40 00052ccb00000000 ffff88083fe81a00 ffff880044636740 000000000000015a 2014-01-07 18:55:40 ffff8800bc659058 ffff881005f91fd8 000000000000fb88 ffff8800bc659058 2014-01-07 18:55:40 Call Trace: 2014-01-07 18:55:40 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:40 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:40 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:40 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:40 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:40 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:40 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:40 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:40 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:40 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:40 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:40 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:40 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:40 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:40 [] kthread+0x96/0xa0 2014-01-07 18:55:40 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io01_0 S 0000000000000004 0 10441 2 0x00000000 2014-01-07 18:55:41 ffff881005f93930 0000000000000046 0000000000000000 ffff881005f938f4 2014-01-07 18:55:41 00052ccb00000000 ffff88083fe81800 ffff88085c456740 00000000000001e0 2014-01-07 18:55:41 ffff8800bc6585f8 ffff881005f93fd8 000000000000fb88 ffff8800bc6585f8 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:41 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:41 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:41 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:41 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:41 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:41 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:41 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:41 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:41 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:41 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io01_0 S 0000000000000004 0 10442 2 0x00000000 2014-01-07 18:55:41 ffff881005f95930 0000000000000046 0000000000000000 ffff881005f958f4 2014-01-07 18:55:41 52ccbe2a00000000 0000000000000286 ffff881005f958d0 ffffffff810810cc 2014-01-07 18:55:41 ffff8800bbcabaf8 ffff881005f95fd8 000000000000fb88 ffff8800bbcabaf8 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ? lock_timer_base+0x3c/0x70 2014-01-07 18:55:41 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:41 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:41 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:41 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:41 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:41 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:41 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:41 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:41 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:41 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:41 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io03_0 S 000000000000000e 0 10443 2 0x00000000 2014-01-07 18:55:41 ffff881005f97da0 0000000000000046 0000000000000000 ffff881005f97d64 2014-01-07 18:55:41 ffff881000000000 ffff88083fe82c00 ffff88085c476740 0000000000000181 2014-01-07 18:55:41 ffff8800bbcab098 ffff881005f97fd8 000000000000fb88 ffff8800bbcab098 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io03_0 S 000000000000000d 0 10444 2 0x00000000 2014-01-07 18:55:41 ffff881005f8dda0 0000000000000046 0000000000000010 ffff881002b2e540 2014-01-07 18:55:41 ffff8810065e4000 0000000000000000 0000000000000058 ffff8807d1a359c0 2014-01-07 18:55:41 ffff8800bbcaa638 ffff881005f8dfd8 000000000000fb88 ffff8800bbcaa638 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:41 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:41 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io02_0 S 0000000000000008 0 10445 2 0x00000000 2014-01-07 18:55:41 ffff881005f8fda0 0000000000000046 0000000000000000 ffff880e83ce2c80 2014-01-07 18:55:41 ffff881031875c00 0000000000000000 0000000000000024 ffff8807d1a359c0 2014-01-07 18:55:41 ffff88007f229ab8 ffff881005f8ffd8 000000000000fb88 ffff88007f229ab8 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io03_0 S 000000000000000d 0 10446 2 0x00000000 2014-01-07 18:55:41 ffff881005e11da0 0000000000000046 ffff881005e11d68 ffff881005e11d64 2014-01-07 18:55:41 ffff881000000000 ffff88083fe82a00 ffff88085c4f6740 0000000000000400 2014-01-07 18:55:41 ffff88007f229058 ffff881005e11fd8 000000000000fb88 ffff88007f229058 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io03_0 S 000000000000000d 0 10447 2 0x00000000 2014-01-07 18:55:41 ffff881005e13da0 0000000000000046 ffff881005e13d68 ffff881005e13d64 2014-01-07 18:55:41 ffff881000000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:41 ffff88007f2285f8 ffff881005e13fd8 000000000000fb88 ffff88007f2285f8 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io00_0 S 0000000000000000 0 10448 2 0x00000000 2014-01-07 18:55:41 ffff881005e35930 0000000000000046 0000000000000000 ffff881005e358f4 2014-01-07 18:55:41 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:41 ffff88007f1b3af8 ffff881005e35fd8 000000000000fb88 ffff88007f1b3af8 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:41 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:41 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:41 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:41 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:41 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:41 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:41 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:41 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:41 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:41 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io00_0 S 0000000000000000 0 10449 2 0x00000000 2014-01-07 18:55:41 ffff881005e37930 0000000000000046 0000000000000000 ffff881005e378f4 2014-01-07 18:55:41 0000000000000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:41 ffff88007f1b3098 ffff881005e37fd8 000000000000fb88 ffff88007f1b3098 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:41 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:41 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:41 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:41 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:41 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:41 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:41 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:41 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:41 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:41 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:41 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:41 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:41 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:41 [] kthread+0x96/0xa0 2014-01-07 18:55:41 [] child_rip+0xa/0x20 2014-01-07 18:55:41 [] ? kthread+0x0/0xa0 2014-01-07 18:55:41 [] ? child_rip+0x0/0x20 2014-01-07 18:55:41 ll_ost_io03_0 S 000000000000000d 0 10450 2 0x00000000 2014-01-07 18:55:41 ffff881005e81930 0000000000000046 ffff881005e818f8 000000003c9cf786 2014-01-07 18:55:41 ffff881000000000 000000000089c000 3331313033303030 6666666638383038 2014-01-07 18:55:41 0000000000000010 0000000000000000 ffff881005e819c0 ffffffff8128093d 2014-01-07 18:55:41 Call Trace: 2014-01-07 18:55:41 [] ? pointer+0x8d/0x830 2014-01-07 18:55:41 [] ? pointer+0x8d/0x830 2014-01-07 18:55:41 [] ? put_dec+0x10c/0x110 2014-01-07 18:55:41 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:41 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:41 [] ? put_dec+0x10c/0x110 2014-01-07 18:55:41 [] ? number+0x2ee/0x320 2014-01-07 18:55:41 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:41 [] ? read_tsc+0x9/0x20 2014-01-07 18:55:41 [] ? vsnprintf+0x336/0x5e0 2014-01-07 18:55:41 [] ? cfs_set_ptldebug_header+0x2b/0xc0 [libcfs] 2014-01-07 18:55:41 [] ? libcfs_debug_vmsg2+0x50b/0xbc0 [libcfs] 2014-01-07 18:55:42 [] ? perf_event_task_sched_out+0x33/0x80 2014-01-07 18:55:42 [] ? dequeue_entity+0x113/0x2e0 2014-01-07 18:55:42 [] ? __dequeue_entity+0x30/0x50 2014-01-07 18:55:42 [] ? pick_next_task_fair+0xd0/0x130 2014-01-07 18:55:42 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:42 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io00_0 S 0000000000000003 0 10451 2 0x00000000 2014-01-07 18:55:42 ffff881005e83da0 0000000000000046 0000000000000000 ffff881005e83d64 2014-01-07 18:55:42 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:42 ffff8800bd675ab8 ffff881005e83fd8 000000000000fb88 ffff8800bd675ab8 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io03_0 S 000000000000000d 0 10452 2 0x00000000 2014-01-07 18:55:42 ffff881005411930 0000000000000046 ffff8810054118f8 ffff8810054118f4 2014-01-07 18:55:42 52ccbe0a00000000 ffff88083fe82a00 ffff880044616740 0000000000000217 2014-01-07 18:55:42 ffff8800bd675058 ffff881005411fd8 000000000000fb88 ffff8800bd675058 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:42 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:42 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:42 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:42 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:42 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:42 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:42 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:42 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:42 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:42 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io00_0 S 0000000000000002 0 10453 2 0x00000000 2014-01-07 18:55:42 ffff881005413930 0000000000000046 0000000000000000 ffff8810054138f4 2014-01-07 18:55:42 00052ccb00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:42 ffff8800bd597af8 ffff881005413fd8 000000000000fb88 ffff8800bd597af8 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:42 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:42 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:42 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:42 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:42 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:42 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:42 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:42 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:42 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:42 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io03_0 S 000000000000000c 0 10454 2 0x00000000 2014-01-07 18:55:42 ffff880e7f4a1930 0000000000000046 ffff880e7f4a18f8 ffff880e7f4a18f4 2014-01-07 18:55:42 52ccbdfd00000000 ffff88083fe82800 ffff880044616740 0000000000000217 2014-01-07 18:55:42 ffff8800bd597098 ffff880e7f4a1fd8 000000000000fb88 ffff8800bd597098 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:42 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:42 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:42 [] ? cfs_crypto_hash_final+0x40/0x80 [libcfs] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:42 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:42 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:42 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:42 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:42 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:42 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:42 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io03_0 S 000000000000000f 0 10455 2 0x00000000 2014-01-07 18:55:42 ffff880e7f4a3da0 0000000000000046 0000000000000000 ffff880e7f4a3d64 2014-01-07 18:55:42 ffff881000000000 ffff88083fe82e00 ffff88085c476740 000000000000015a 2014-01-07 18:55:42 ffff8800bd596638 ffff880e7f4a3fd8 000000000000fb88 ffff8800bd596638 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io02_0 S 0000000000000009 0 10456 2 0x00000000 2014-01-07 18:55:42 ffff880e7f595da0 0000000000000046 0000000000000000 ffff880e7f595d64 2014-01-07 18:55:42 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:42 ffff8800ba99fab8 ffff880e7f595fd8 000000000000fb88 ffff8800ba99fab8 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io02_0 S 0000000000000008 0 10457 2 0x00000000 2014-01-07 18:55:42 ffff880e7f597da0 0000000000000046 0000000000000000 ffff880e7f597d64 2014-01-07 18:55:42 ffff881000000000 ffff88083fe82000 ffff880044636740 00000000000001ff 2014-01-07 18:55:42 ffff8800ba99f058 ffff880e7f597fd8 000000000000fb88 ffff8800ba99f058 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io02_0 S 0000000000000009 0 10458 2 0x00000000 2014-01-07 18:55:42 ffff880e80b09da0 0000000000000046 0000000000000000 ffff8810058c3140 2014-01-07 18:55:42 ffff881031875c00 0000000000000000 0000000000000008 ffff8807d1a359c0 2014-01-07 18:55:42 ffff8800ba99e5f8 ffff880e80b09fd8 000000000000fb88 ffff8800ba99e5f8 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io00_0 S 0000000000000000 0 10459 2 0x00000000 2014-01-07 18:55:42 ffff880e80b0bda0 0000000000000046 0000000000000000 ffff880e80b0bd64 2014-01-07 18:55:42 ffff880700000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:42 ffff8800bc6efaf8 ffff880e80b0bfd8 000000000000fb88 ffff8800bc6efaf8 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io00_0 S 0000000000000002 0 10460 2 0x00000000 2014-01-07 18:55:42 ffff880e815f1da0 0000000000000046 0000000000000000 ffff880e815f1d64 2014-01-07 18:55:42 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:42 ffff8800bc6ef098 ffff880e815f1fd8 000000000000fb88 ffff8800bc6ef098 2014-01-07 18:55:42 Call Trace: 2014-01-07 18:55:42 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:42 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:42 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:42 [] kthread+0x96/0xa0 2014-01-07 18:55:42 [] child_rip+0xa/0x20 2014-01-07 18:55:42 [] ? kthread+0x0/0xa0 2014-01-07 18:55:42 [] ? child_rip+0x0/0x20 2014-01-07 18:55:42 ll_ost_io00_0 S 0000000000000002 0 10461 2 0x00000000 2014-01-07 18:55:42 ffff880e815f3da0 0000000000000046 0000000000000010 ffff88007f877a40 2014-01-07 18:55:42 ffff8807c0202400 0000000000000000 0000000000000016 ffff8807d1a359c0 2014-01-07 18:55:42 ffff8800bd6745f8 ffff880e815f3fd8 000000000000fb88 ffff8800bd6745f8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:43 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io02_0 S 0000000000000008 0 10462 2 0x00000000 2014-01-07 18:55:43 ffff880e815f5da0 0000000000000046 ffff880e815f5d68 ffff880e815f5d64 2014-01-07 18:55:43 ffff881000000000 ffff88083fe82000 ffff880044616740 0000000000000207 2014-01-07 18:55:43 ffff8800bd62dab8 ffff880e815f5fd8 000000000000fb88 ffff8800bd62dab8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io02_0 S 0000000000000008 0 10463 2 0x00000000 2014-01-07 18:55:43 ffff880e80849da0 0000000000000046 ffff880e80849d68 ffff880e80849d64 2014-01-07 18:55:43 ffff881031875c00 ffff88083fe82000 ffff88085c456740 0000000000000400 2014-01-07 18:55:43 ffff8800bd62d058 ffff880e80849fd8 000000000000fb88 ffff8800bd62d058 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000003 0 10464 2 0x00000000 2014-01-07 18:55:43 ffff880e8084b930 0000000000000046 0000000000000000 ffff880e8084b8f4 2014-01-07 18:55:43 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff8800bd62c5f8 ffff880e8084bfd8 000000000000fb88 ffff8800bd62c5f8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:43 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:43 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:43 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:43 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:43 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:43 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:43 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:43 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:43 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:43 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:43 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000002 0 10465 2 0x00000000 2014-01-07 18:55:43 ffff880e81741930 0000000000000046 0000000000000000 ffff880e817418f4 2014-01-07 18:55:43 ffff880e00000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff88007f1bdaf8 ffff880e81741fd8 000000000000fb88 ffff88007f1bdaf8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:43 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:43 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:43 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:43 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:43 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:43 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:43 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:43 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:43 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:43 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000003 0 10466 2 0x00000000 2014-01-07 18:55:43 ffff880e81743da0 0000000000000046 0000000000000000 ffff880e81743d64 2014-01-07 18:55:43 ffff880700000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff88007f1bd098 ffff880e81743fd8 000000000000fb88 ffff88007f1bd098 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000000 0 10467 2 0x00000000 2014-01-07 18:55:43 ffff880e81745930 0000000000000046 0000000000000000 ffff880e817458f4 2014-01-07 18:55:43 00052ccb00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff8800bd5f1ab8 ffff880e81745fd8 000000000000fb88 ffff8800bd5f1ab8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:43 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:43 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:43 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:43 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:43 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:43 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:43 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:43 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:43 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:43 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io02_0 S 0000000000000008 0 10468 2 0x00000000 2014-01-07 18:55:43 ffff880e81747da0 0000000000000046 0000000000000000 ffff880e81747d64 2014-01-07 18:55:43 ffff881000000000 ffff88083fe82000 ffff88085c476740 000000000000015a 2014-01-07 18:55:43 ffff8800bd5f1058 ffff880e81747fd8 000000000000fb88 ffff8800bd5f1058 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io02_0 S 000000000000000a 0 10469 2 0x00000000 2014-01-07 18:55:43 ffff880e7eefdda0 0000000000000046 0000000000000000 ffff880e8fc28b40 2014-01-07 18:55:43 ffff881031875c00 0000000000000000 0000000000000045 ffff8807d1a359c0 2014-01-07 18:55:43 ffff8800bd5f05f8 ffff880e7eefdfd8 000000000000fb88 ffff8800bd5f05f8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000002 0 10470 2 0x00000000 2014-01-07 18:55:43 ffff880e7eeffda0 0000000000000046 0000000000000000 ffff880e7eeffd64 2014-01-07 18:55:43 ffff880700000000 ffff88083fe81400 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff8800bc887af8 ffff880e7eefffd8 000000000000fb88 ffff8800bc887af8 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io02_0 S 0000000000000009 0 10471 2 0x00000000 2014-01-07 18:55:43 ffff880e7f015da0 0000000000000046 0000000000000000 ffff880e7f015d64 2014-01-07 18:55:43 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:43 ffff8800bc887098 ffff880e7f015fd8 000000000000fb88 ffff8800bc887098 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:43 [] kthread+0x96/0xa0 2014-01-07 18:55:43 [] child_rip+0xa/0x20 2014-01-07 18:55:43 [] ? kthread+0x0/0xa0 2014-01-07 18:55:43 [] ? child_rip+0x0/0x20 2014-01-07 18:55:43 ll_ost_io00_0 S 0000000000000003 0 10472 2 0x00000000 2014-01-07 18:55:43 ffff880e7f017930 0000000000000046 0000000000000000 ffff880e7f0178f4 2014-01-07 18:55:43 00052ccb00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:43 ffff8800bc886638 ffff880e7f017fd8 000000000000fb88 ffff8800bc886638 2014-01-07 18:55:43 Call Trace: 2014-01-07 18:55:43 [] schedule_timeout+0x192/0x2e0 2014-01-07 18:55:43 [] ? process_timeout+0x0/0x10 2014-01-07 18:55:43 [] target_bulk_io+0x3a0/0x8d0 [ptlrpc] 2014-01-07 18:55:43 [] ? cfs_crypto_hash_final+0x48/0x80 [libcfs] 2014-01-07 18:55:43 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:43 [] ost_brw_read+0x103c/0x1350 [ost] 2014-01-07 18:55:43 [] ? target_bulk_timeout+0x0/0xc0 [ptlrpc] 2014-01-07 18:55:43 [] ? libcfs_debug_msg+0x41/0x50 [libcfs] 2014-01-07 18:55:43 [] ost_handle+0x24a8/0x44d0 [ost] 2014-01-07 18:55:43 [] ? libcfs_id2str+0x74/0xb0 [libcfs] 2014-01-07 18:55:44 [] ptlrpc_server_handle_request+0x385/0xc00 [ptlrpc] 2014-01-07 18:55:44 [] ? lc_watchdog_touch+0xd5/0x170 [libcfs] 2014-01-07 18:55:44 [] ? ptlrpc_wait_event+0xa9/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0xaed/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io03_0 S 000000000000000e 0 10473 2 0x00000000 2014-01-07 18:55:44 ffff880e7f47dda0 0000000000000046 ffff880e7f47dd68 ffff880e7f47dd64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82c00 ffff88085c4b6740 0000000000000400 2014-01-07 18:55:44 ffff8800bd181ab8 ffff880e7f47dfd8 000000000000fb88 ffff8800bd181ab8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000009 0 10474 2 0x00000000 2014-01-07 18:55:44 ffff880e7f47fda0 0000000000000046 0000000000000000 ffff880e7f47fd64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:44 ffff8800bd181058 ffff880e7f47ffd8 000000000000fb88 ffff8800bd181058 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io03_0 S 000000000000000e 0 10475 2 0x00000000 2014-01-07 18:55:44 ffff880e7b8c1da0 0000000000000046 ffff880e7b8c1d68 ffff880e7b8c1d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82c00 ffff880044616740 0000000000000217 2014-01-07 18:55:44 ffff8800bd1805f8 ffff880e7b8c1fd8 000000000000fb88 ffff8800bd1805f8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000008 0 10476 2 0x00000000 2014-01-07 18:55:44 ffff880e7b8c3da0 0000000000000046 0000000000000000 ffff881002fa6b00 2014-01-07 18:55:44 ffff881031875c00 0000000000000000 000000000000005a ffff8807d1a359c0 2014-01-07 18:55:44 ffff88007f285af8 ffff880e7b8c3fd8 000000000000fb88 ffff88007f285af8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 000000000000000b 0 10477 2 0x00000000 2014-01-07 18:55:44 ffff880e7f495da0 0000000000000046 0000000000000000 ffff880e82680840 2014-01-07 18:55:44 ffff881031875c00 0000000000000000 0000000000000044 ffff8807d1a359c0 2014-01-07 18:55:44 ffff88007f285098 ffff880e7f495fd8 000000000000fb88 ffff88007f285098 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 000000000000000a 0 10478 2 0x00000000 2014-01-07 18:55:44 ffff880e7f497da0 0000000000000046 0000000000000000 ffff880e7f497d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82400 ffff880044636740 00000000000001ff 2014-01-07 18:55:44 ffff88007f284638 ffff880e7f497fd8 000000000000fb88 ffff88007f284638 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io03_0 S 000000000000000e 0 10479 2 0x00000000 2014-01-07 18:55:44 ffff881005801da0 0000000000000046 0000000000000000 ffff881005801d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82c00 ffff88085c476740 0000000000000181 2014-01-07 18:55:44 ffff8800bc201ab8 ffff881005801fd8 000000000000fb88 ffff8800bc201ab8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000009 0 10480 2 0x00000000 2014-01-07 18:55:44 ffff881005803da0 0000000000000046 0000000000000000 ffff881005803d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82200 ffff88085c476740 0000000000000181 2014-01-07 18:55:44 ffff88007f1bc638 ffff881005803fd8 000000000000fb88 ffff88007f1bc638 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000009 0 10481 2 0x00000000 2014-01-07 18:55:44 ffff881005955da0 0000000000000046 0000000000000000 ffff881005955d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:44 ffff88007f35daf8 ffff881005955fd8 000000000000fb88 ffff88007f35daf8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000009 0 10482 2 0x00000000 2014-01-07 18:55:44 ffff881005957da0 0000000000000046 ffff881005957d68 ffff881005957d64 2014-01-07 18:55:44 ffff881031875c00 ffff88083fe82200 ffff88085c416740 0000000000000400 2014-01-07 18:55:44 ffff88007f35d098 ffff881005957fd8 000000000000fb88 ffff88007f35d098 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000009 0 10483 2 0x00000000 2014-01-07 18:55:44 ffff881005fa5da0 0000000000000046 0000000000000000 ffff881005fa5d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82200 ffff88085c476740 0000000000000181 2014-01-07 18:55:44 ffff88007f35c638 ffff881005fa5fd8 000000000000fb88 ffff88007f35c638 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 000000000000000a 0 10484 2 0x00000000 2014-01-07 18:55:44 ffff881005fa7da0 0000000000000046 0000000000000000 ffff881005fa7d64 2014-01-07 18:55:44 ffff881000000000 ffff88083fe82400 ffff88085c476740 000000000000015a 2014-01-07 18:55:44 ffff8800bd683ab8 ffff881005fa7fd8 000000000000fb88 ffff8800bd683ab8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 0000000000000008 0 10485 2 0x00000000 2014-01-07 18:55:44 ffff880e7f7b1da0 0000000000000046 0000000000000000 ffff881002d4da40 2014-01-07 18:55:44 ffff881031875c00 0000000000000000 0000000000000014 ffff8807d1a359c0 2014-01-07 18:55:44 ffff8800bd683058 ffff880e7f7b1fd8 000000000000fb88 ffff8800bd683058 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:44 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:44 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:44 [] kthread+0x96/0xa0 2014-01-07 18:55:44 [] child_rip+0xa/0x20 2014-01-07 18:55:44 [] ? kthread+0x0/0xa0 2014-01-07 18:55:44 [] ? child_rip+0x0/0x20 2014-01-07 18:55:44 ll_ost_io02_0 S 000000000000000a 0 10486 2 0x00000000 2014-01-07 18:55:44 ffff880e7f7bbda0 0000000000000046 ffff881000000010 ffff881003b33500 2014-01-07 18:55:44 ffff881031875c00 0000000000000000 000000000000004d ffff8807d1a359c0 2014-01-07 18:55:44 ffff8800bd6825f8 ffff880e7f7bbfd8 000000000000fb88 ffff8800bd6825f8 2014-01-07 18:55:44 Call Trace: 2014-01-07 18:55:44 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:44 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 0000000000000008 0 10487 2 0x00000000 2014-01-07 18:55:45 ffff880e7b805da0 0000000000000046 0000000000000000 ffff880e82680240 2014-01-07 18:55:45 ffff881031875c00 0000000000000000 000000000000005a ffff8807d1a359c0 2014-01-07 18:55:45 ffff88007f19faf8 ffff880e7b805fd8 000000000000fb88 ffff88007f19faf8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 000000000000000a 0 10488 2 0x00000000 2014-01-07 18:55:45 ffff880e7b80fda0 0000000000000046 0000000000000010 ffff88100345f240 2014-01-07 18:55:45 ffff881031875c00 0000000000000000 0000000000000013 ffff8807d1a359c0 2014-01-07 18:55:45 ffff88007f19f098 ffff880e7b80ffd8 000000000000fb88 ffff88007f19f098 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:45 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 0000000000000009 0 10489 2 0x00000000 2014-01-07 18:55:45 ffff880e7efb9da0 0000000000000046 0000000000000000 ffff880e813d39c0 2014-01-07 18:55:45 ffff881031875c00 0000000000000000 000000000000001b ffff8807d1a359c0 2014-01-07 18:55:45 ffff88007f19e638 ffff880e7efb9fd8 000000000000fb88 ffff88007f19e638 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 0000000000000009 0 10490 2 0x00000000 2014-01-07 18:55:45 ffff880e7f633da0 0000000000000046 0000000000000000 ffff880e7f633d64 2014-01-07 18:55:45 ffff881000000000 ffff88083fe82200 ffff88085c476740 000000000000015a 2014-01-07 18:55:45 ffff88007f11fab8 ffff880e7f633fd8 000000000000fb88 ffff88007f11fab8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 0000000000000008 0 10491 2 0x00000000 2014-01-07 18:55:45 ffff880e817adda0 0000000000000046 0000000000000000 ffff881002d4dd40 2014-01-07 18:55:45 ffff881031875c00 0000000000000000 000000000000003c ffff8807d1a359c0 2014-01-07 18:55:45 ffff88007f11f058 ffff880e817adfd8 000000000000fb88 ffff88007f11f058 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 000000000000000a 0 10492 2 0x00000000 2014-01-07 18:55:45 ffff880e800e5da0 0000000000000046 ffff881000000010 ffff880e82680e40 2014-01-07 18:55:45 ffff881031875c00 0000000000000000 000000000000001a ffff8807d1a359c0 2014-01-07 18:55:45 ffff88007f11e5f8 ffff880e800e5fd8 000000000000fb88 ffff88007f11e5f8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:45 [] ? ptlrpc_server_high_pending+0x51/0xb0 [ptlrpc] 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 ll_ost_io02_0 S 000000000000000a 0 10493 2 0x00000000 2014-01-07 18:55:45 ffff880e80201da0 0000000000000046 ffff880e80201d68 ffff880e80201d64 2014-01-07 18:55:45 ffff881031875c00 ffff88083fe82400 ffff88085c416740 0000000000000207 2014-01-07 18:55:45 ffff88007f9c5af8 ffff880e80201fd8 000000000000fb88 ffff88007f9c5af8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ptlrpc_wait_event+0x2c5/0x2d0 [ptlrpc] 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ptlrpc_main+0x81f/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] ? ptlrpc_main+0x0/0x1740 [ptlrpc] 2014-01-07 18:55:45 [] kthread+0x96/0xa0 2014-01-07 18:55:45 [] child_rip+0xa/0x20 2014-01-07 18:55:45 [] ? kthread+0x0/0xa0 2014-01-07 18:55:45 [] ? child_rip+0x0/0x20 2014-01-07 18:55:45 in.mrlogind S 0000000000000009 0 11462 5188 0x00000000 2014-01-07 18:55:45 ffff880e7b711908 0000000000000086 ffff880e7b7118d0 ffff880e7b7118cc 2014-01-07 18:55:45 dead000000000000 ffff88083fe82200 ffff880044616740 0000000000000217 2014-01-07 18:55:45 ffff88007f9c5098 ffff880e7b711fd8 000000000000fb88 ffff88007f9c5098 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:55:45 [] ? flush_delayed_work+0x54/0x70 2014-01-07 18:55:45 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:55:45 [] do_select+0x57c/0x6c0 2014-01-07 18:55:45 [] ? __pollwait+0x0/0xf0 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:45 [] ? release_sock+0xce/0xe0 2014-01-07 18:55:45 [] ? tcp_sendmsg+0x73c/0xa20 2014-01-07 18:55:45 [] ? sock_aio_write+0x19b/0x1c0 2014-01-07 18:55:45 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:55:45 [] ? n_tty_read+0x34c/0x9b0 2014-01-07 18:55:45 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:45 [] sys_select+0x47/0x110 2014-01-07 18:55:45 [] ? sys_write+0x51/0x90 2014-01-07 18:55:45 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:45 login S 0000000000000003 0 11463 11462 0x00000000 2014-01-07 18:55:45 ffff880e7b771e78 0000000000000082 ffffffff81065d54 0000000000000029 2014-01-07 18:55:45 ffff8810308dc840 00007ffff7ffd978 ffff881030f7fe08 0000000000000007 2014-01-07 18:55:45 ffff8800bb827ab8 ffff880e7b771fd8 000000000000fb88 ffff8800bb827ab8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:55:45 [] do_wait+0x1e4/0x240 2014-01-07 18:55:45 [] sys_wait4+0xa3/0x100 2014-01-07 18:55:45 [] ? child_wait_callback+0x0/0x70 2014-01-07 18:55:45 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:45 bash S 0000000000000003 0 11464 11463 0x00000000 2014-01-07 18:55:45 ffff880e7b7d1e78 0000000000000082 0000000000000000 0000000000000286 2014-01-07 18:55:45 ffff880e7b7d1df8 ffffffff813240b2 ffff88007f37a000 00007fffffffeffd 2014-01-07 18:55:45 ffff8800bb827058 ffff880e7b7d1fd8 000000000000fb88 ffff8800bb827058 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] ? tty_get_pgrp+0x42/0x50 2014-01-07 18:55:45 [] do_wait+0x1e4/0x240 2014-01-07 18:55:45 [] sys_wait4+0xa3/0x100 2014-01-07 18:55:45 [] ? child_wait_callback+0x0/0x70 2014-01-07 18:55:45 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:45 top S 0000000000000003 0 11510 11464 0x00000000 2014-01-07 18:55:45 ffff880e7b1b7908 0000000000000086 0000000000000000 ffff880e7b1b78cc 2014-01-07 18:55:45 ffff880e00000000 ffff88083fe81600 ffff880044636740 000000000000015a 2014-01-07 18:55:45 ffff8800bb8265f8 ffff880e7b1b7fd8 000000000000fb88 ffff8800bb8265f8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] schedule_hrtimeout_range+0xc8/0x160 2014-01-07 18:55:45 [] ? hrtimer_wakeup+0x0/0x30 2014-01-07 18:55:45 [] ? hrtimer_start_range_ns+0x14/0x20 2014-01-07 18:55:45 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:55:45 [] do_select+0x57c/0x6c0 2014-01-07 18:55:45 [] ? __pollwait+0x0/0xf0 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? enqueue_task_fair+0x64/0x100 2014-01-07 18:55:45 [] ? enqueue_task+0x66/0x80 2014-01-07 18:55:45 [] ? check_preempt_curr+0x6d/0x90 2014-01-07 18:55:45 [] ? try_to_wake_up+0x24e/0x3e0 2014-01-07 18:55:45 [] ? default_wake_function+0x12/0x20 2014-01-07 18:55:45 [] ? pollwake+0x56/0x60 2014-01-07 18:55:45 [] ? default_wake_function+0x0/0x20 2014-01-07 18:55:45 [] ? __wake_up_common+0x59/0x90 2014-01-07 18:55:45 [] ? __wake_up+0x53/0x70 2014-01-07 18:55:45 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:55:45 [] ? tty_ldisc_deref+0xe/0x10 2014-01-07 18:55:45 [] ? tty_ioctl+0xf7/0x950 2014-01-07 18:55:45 [] ? vfs_ioctl+0x22/0xa0 2014-01-07 18:55:45 [] ? read_tsc+0x9/0x20 2014-01-07 18:55:45 [] ? ktime_get_ts+0xb1/0xf0 2014-01-07 18:55:45 [] sys_select+0x47/0x110 2014-01-07 18:55:45 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:45 in.mrlogind S 0000000000000000 0 11551 5188 0x00000000 2014-01-07 18:55:45 ffff880e7acf1908 0000000000000086 0000000000000000 ffff880e7acf18cc 2014-01-07 18:55:45 ffff880e00000000 ffff88083fe81000 ffff880044636740 000000000000015a 2014-01-07 18:55:45 ffff8800bc2005f8 ffff880e7acf1fd8 000000000000fb88 ffff8800bc2005f8 2014-01-07 18:55:45 Call Trace: 2014-01-07 18:55:45 [] schedule_hrtimeout_range+0x13d/0x160 2014-01-07 18:55:45 [] ? flush_delayed_work+0x54/0x70 2014-01-07 18:55:45 [] poll_schedule_timeout+0x39/0x60 2014-01-07 18:55:45 [] do_select+0x57c/0x6c0 2014-01-07 18:55:45 [] ? __pollwait+0x0/0xf0 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:45 [] ? pollwake+0x0/0x60 2014-01-07 18:55:46 [] ? _spin_unlock_bh+0x1b/0x20 2014-01-07 18:55:46 [] ? release_sock+0xce/0xe0 2014-01-07 18:55:46 [] ? tcp_sendmsg+0x73c/0xa20 2014-01-07 18:55:46 [] ? sock_aio_write+0x19b/0x1c0 2014-01-07 18:55:46 [] core_sys_select+0x18a/0x2c0 2014-01-07 18:55:46 [] ? n_tty_read+0x34c/0x9b0 2014-01-07 18:55:46 [] ? autoremove_wake_function+0x0/0x40 2014-01-07 18:55:46 [] sys_select+0x47/0x110 2014-01-07 18:55:46 [] ? sys_write+0x51/0x90 2014-01-07 18:55:46 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:46 login S 0000000000000003 0 11552 11551 0x00000000 2014-01-07 18:55:46 ffff880e7adb5e78 0000000000000082 0000000000000000 0000000000000029 2014-01-07 18:55:46 ffff8810319d8740 00007ffff7ffd978 ffff88103093a0c0 0000000000000007 2014-01-07 18:55:46 ffff8800bcd7daf8 ffff880e7adb5fd8 000000000000fb88 ffff8800bcd7daf8 2014-01-07 18:55:46 Call Trace: 2014-01-07 18:55:46 [] do_wait+0x1e4/0x240 2014-01-07 18:55:46 [] sys_wait4+0xa3/0x100 2014-01-07 18:55:46 [] ? child_wait_callback+0x0/0x70 2014-01-07 18:55:46 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:46 bash R running task 0 11553 11552 0x00000008 2014-01-07 18:55:46 ffffffff81c07600 ffffffff8106ecff ffff880e7adffc38 ffff880e7adffc8d 2014-01-07 18:55:46 ffffffff81eacf01 0000000000000014 ffff880e7adffc78 0000000000000034 2014-01-07 18:55:46 ffff880e7adffce8 0000000000000046 ffff880e7adffc78 0000000000000046 2014-01-07 18:55:46 Call Trace: 2014-01-07 18:55:46 [] ? print_context_stack+0x85/0x140 2014-01-07 18:55:46 [] ? dump_trace+0x190/0x3b0 2014-01-07 18:55:46 [] show_trace_log_lvl+0x55/0x70 2014-01-07 18:55:46 [] show_stack_log_lvl+0x9a/0x170 2014-01-07 18:55:46 [] show_stack+0x1a/0x20 2014-01-07 18:55:46 [] sched_show_task+0x98/0x100 2014-01-07 18:55:46 [] show_state_filter+0x7b/0xc0 2014-01-07 18:55:46 [] sysrq_handle_showstate+0x10/0x20 2014-01-07 18:55:46 [] __handle_sysrq+0x132/0x1a0 2014-01-07 18:55:46 [] write_sysrq_trigger+0x4e/0x50 2014-01-07 18:55:46 [] proc_reg_write+0x7e/0xc0 2014-01-07 18:55:46 [] vfs_write+0xb8/0x1a0 2014-01-07 18:55:46 [] sys_write+0x51/0x90 2014-01-07 18:55:46 [] system_call_fastpath+0x16/0x1b 2014-01-07 18:55:46 Sched Debug Version: v0.09, 2.6.32-358.18.1.el6_lustre.gf1706f8.x86_64 #1 2014-01-07 18:55:46 now at 5726472.863821 msecs 2014-01-07 18:55:46 .jiffies : 4300393779 2014-01-07 18:55:46 .sysctl_sched_latency : 20.000000 2014-01-07 18:55:46 .sysctl_sched_min_granularity : 4.000000 2014-01-07 18:55:46 .sysctl_sched_wakeup_granularity : 4.000000 2014-01-07 18:55:46 .sysctl_sched_child_runs_first : 0.000000 2014-01-07 18:55:46 .sysctl_sched_features : 3183 2014-01-07 18:55:46 .sysctl_sched_tunable_scaling : 1 (logaritmic) 2014-01-07 18:55:46 2014-01-07 18:55:46 cpu#0, 2593.668 MHz 2014-01-07 18:55:46 .nr_running : 5 2014-01-07 18:55:46 .load : 1069 2014-01-07 18:55:46 .nr_switches : 26220551 2014-01-07 18:55:46 .nr_load_updates : 918412 2014-01-07 18:55:46 .nr_uninterruptible : 0 2014-01-07 18:55:46 .next_balance : 4300.349402 2014-01-07 18:55:46 .curr->pid : 5188 2014-01-07 18:55:46 .clock : 5671717.291208 2014-01-07 18:55:46 .cpu_load[0] : 0 2014-01-07 18:55:46 .cpu_load[1] : 0 2014-01-07 18:55:46 .cpu_load[2] : 0 2014-01-07 18:55:46 .cpu_load[3] : 0 2014-01-07 18:55:46 .cpu_load[4] : 0 2014-01-07 18:55:46 .yld_count : 0 2014-01-07 18:55:46 .sched_switch : 0 2014-01-07 18:55:46 .sched_count : 31876715 2014-01-07 18:55:46 .sched_goidle : 11535108 2014-01-07 18:55:46 .avg_idle : 1000000 2014-01-07 18:55:46 .ttwu_count : 14601451 2014-01-07 18:55:46 .ttwu_local : 3057626 2014-01-07 18:55:46 .bkl_count : 1242 2014-01-07 18:55:46 2014-01-07 18:55:46 cfs_rq[0]:/ 2014-01-07 18:55:46 .exec_clock : 492649.425787 2014-01-07 18:55:46 .MIN_vruntime : 9260784.823018 2014-01-07 18:55:46 .min_vruntime : 9260794.823018 2014-01-07 18:55:46 .max_vruntime : 9260794.823018 2014-01-07 18:55:46 .spread : 10.000000 2014-01-07 18:55:46 .spread0 : 0.000000 2014-01-07 18:55:46 .nr_spread_over : 5051 2014-01-07 18:55:46 .nr_running : 4 2014-01-07 18:55:46 .load : 1069 2014-01-07 18:55:46 .load_avg : 0.000000 2014-01-07 18:55:46 .load_period : 0.000000 2014-01-07 18:55:46 .load_contrib : 0 2014-01-07 18:55:46 .load_tg : 0 2014-01-07 18:55:46 2014-01-07 18:55:46 rt_rq[0]:/ 2014-01-07 18:55:46 .rt_nr_running : 1 2014-01-07 18:55:46 .rt_throttled : 0 2014-01-07 18:55:46 .rt_time : 0.000000 2014-01-07 18:55:46 .rt_runtime : 950.000000 2014-01-07 18:55:46 2014-01-07 18:55:46 runnable tasks: 2014-01-07 18:55:46 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:46 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:46 migration/0 3 175.952543 228 0 175.952543 391.636871 0.000000 / 2014-01-07 18:55:46 R xinetd 5188 9260821.752220 215 120 9260821.752220 82.252712 5583514.229236 / 2014-01-07 18:55:46 spl_kmem_cache/ 5399 9260784.823018 52463 139 9260784.823018 6141.590591 5573275.693863 / 2014-01-07 18:55:46 z_null_iss/0 6744 9260784.823018 199008 139 9260784.823018 1283.502345 5390802.664525 / 2014-01-07 18:55:46 z_rd_int/0 6754 9260794.823018 681656 139 9260794.823018 37316.775587 5358197.676327 / 2014-01-07 18:55:46 2014-01-07 18:55:46 cpu#1, 2593.668 MHz 2014-01-07 18:55:46 .nr_running : 3 2014-01-07 18:55:46 .load : 1039 2014-01-07 18:55:46 .nr_switches : 22522255 2014-01-07 18:55:46 .nr_load_updates : 1353231 2014-01-07 18:55:46 .nr_uninterruptible : 0 2014-01-07 18:55:46 .next_balance : 4300.291999 2014-01-07 18:55:46 .curr->pid : 11553 2014-01-07 18:55:46 .clock : 5615020.716835 2014-01-07 18:55:46 .cpu_load[0] : 0 2014-01-07 18:55:46 .cpu_load[1] : 2 2014-01-07 18:55:46 .cpu_load[2] : 2 2014-01-07 18:55:46 .cpu_load[3] : 0 2014-01-07 18:55:46 .cpu_load[4] : 2 2014-01-07 18:55:46 .yld_count : 0 2014-01-07 18:55:46 .sched_switch : 0 2014-01-07 18:55:46 .sched_count : 29902534 2014-01-07 18:55:46 .sched_goidle : 9764351 2014-01-07 18:55:46 .avg_idle : 1000000 2014-01-07 18:55:46 .ttwu_count : 12696266 2014-01-07 18:55:46 .ttwu_local : 3745581 2014-01-07 18:55:46 .bkl_count : 175 2014-01-07 18:55:46 2014-01-07 18:55:46 cfs_rq[1]:/ 2014-01-07 18:55:46 .exec_clock : 469224.123274 2014-01-07 18:55:46 .MIN_vruntime : 8862955.211303 2014-01-07 18:55:46 .min_vruntime : 8862955.211303 2014-01-07 18:55:46 .max_vruntime : 8862955.211303 2014-01-07 18:55:46 .spread : 0.000000 2014-01-07 18:55:46 .spread0 : -397839.611715 2014-01-07 18:55:46 .nr_spread_over : 3891 2014-01-07 18:55:46 .nr_running : 2 2014-01-07 18:55:46 .load : 1039 2014-01-07 18:55:46 .load_avg : 0.000000 2014-01-07 18:55:46 .load_period : 0.000000 2014-01-07 18:55:46 .load_contrib : 0 2014-01-07 18:55:46 .load_tg : 0 2014-01-07 18:55:46 2014-01-07 18:55:46 rt_rq[1]:/ 2014-01-07 18:55:46 .rt_nr_running : 1 2014-01-07 18:55:46 .rt_throttled : 0 2014-01-07 18:55:46 .rt_time : 0.000000 2014-01-07 18:55:46 .rt_runtime : 950.000000 2014-01-07 18:55:46 2014-01-07 18:55:46 runnable tasks: 2014-01-07 18:55:46 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:46 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:46 migration/1 7 187.260105 1399 0 187.260105 332.337723 0.000000 / 2014-01-07 18:55:46 z_rd_int/1 6755 8862955.211303 665829 139 8862955.211303 38284.272280 5301817.217475 / 2014-01-07 18:55:46 R bash 11553 8863332.392084 75 120 8863332.392084 401.092028 14061.496477 / 2014-01-07 18:55:46 2014-01-07 18:55:46 cpu#2, 2593.668 MHz 2014-01-07 18:55:46 .nr_running : 0 2014-01-07 18:55:46 .load : 0 2014-01-07 18:55:46 .nr_switches : 18062121 2014-01-07 18:55:46 .nr_load_updates : 997094 2014-01-07 18:55:46 .nr_uninterruptible : 4 2014-01-07 18:55:46 .next_balance : 4300.394742 2014-01-07 18:55:46 .curr->pid : 0 2014-01-07 18:55:46 .clock : 5716810.843465 2014-01-07 18:55:46 .cpu_load[0] : 0 2014-01-07 18:55:46 .cpu_load[1] : 0 2014-01-07 18:55:46 .cpu_load[2] : 0 2014-01-07 18:55:46 .cpu_load[3] : 0 2014-01-07 18:55:46 .cpu_load[4] : 1 2014-01-07 18:55:46 .yld_count : 0 2014-01-07 18:55:46 .sched_switch : 0 2014-01-07 18:55:46 .sched_count : 19810320 2014-01-07 18:55:46 .sched_goidle : 7433442 2014-01-07 18:55:46 .avg_idle : 818898 2014-01-07 18:55:46 .ttwu_count : 10542568 2014-01-07 18:55:46 .ttwu_local : 2890699 2014-01-07 18:55:46 .bkl_count : 259 2014-01-07 18:55:46 2014-01-07 18:55:46 cfs_rq[2]:/ 2014-01-07 18:55:46 .exec_clock : 399961.983871 2014-01-07 18:55:46 .MIN_vruntime : 0.000001 2014-01-07 18:55:46 .min_vruntime : 8142978.243556 2014-01-07 18:55:46 .max_vruntime : 0.000001 2014-01-07 18:55:46 .spread : 0.000000 2014-01-07 18:55:46 .spread0 : -1117816.579462 2014-01-07 18:55:46 .nr_spread_over : 4102 2014-01-07 18:55:46 .nr_running : 0 2014-01-07 18:55:46 .load : 0 2014-01-07 18:55:46 .load_avg : 0.000000 2014-01-07 18:55:46 .load_period : 0.000000 2014-01-07 18:55:46 .load_contrib : 0 2014-01-07 18:55:46 .load_tg : 0 2014-01-07 18:55:46 2014-01-07 18:55:46 rt_rq[2]:/ 2014-01-07 18:55:46 .rt_nr_running : 0 2014-01-07 18:55:46 .rt_throttled : 0 2014-01-07 18:55:46 .rt_time : 0.000000 2014-01-07 18:55:46 .rt_runtime : 950.000000 2014-01-07 18:55:46 2014-01-07 18:55:46 runnable tasks: 2014-01-07 18:55:46 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:46 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:46 2014-01-07 18:55:46 cpu#3, 2593.668 MHz 2014-01-07 18:55:46 .nr_running : 2 2014-01-07 18:55:46 .load : 1024 2014-01-07 18:55:46 .nr_switches : 14766437 2014-01-07 18:55:46 .nr_load_updates : 904411 2014-01-07 18:55:46 .nr_uninterruptible : 0 2014-01-07 18:55:46 .next_balance : 4300.394771 2014-01-07 18:55:46 .curr->pid : 0 2014-01-07 18:55:46 .clock : 5716999.459290 2014-01-07 18:55:46 .cpu_load[0] : 0 2014-01-07 18:55:46 .cpu_load[1] : 0 2014-01-07 18:55:46 .cpu_load[2] : 0 2014-01-07 18:55:46 .cpu_load[3] : 6 2014-01-07 18:55:46 .cpu_load[4] : 60 2014-01-07 18:55:46 .yld_count : 0 2014-01-07 18:55:46 .sched_switch : 0 2014-01-07 18:55:46 .sched_count : 15406357 2014-01-07 18:55:46 .sched_goidle : 5816337 2014-01-07 18:55:46 .avg_idle : 1000000 2014-01-07 18:55:46 .ttwu_count : 8867243 2014-01-07 18:55:46 .ttwu_local : 2104573 2014-01-07 18:55:47 .bkl_count : 97 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[3]:/ 2014-01-07 18:55:47 .exec_clock : 342391.386710 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 8351939.503281 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -908855.319737 2014-01-07 18:55:47 .nr_spread_over : 4406 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .load_avg : 0.000000 2014-01-07 18:55:47 .load_period : 0.000000 2014-01-07 18:55:47 .load_contrib : 0 2014-01-07 18:55:47 .load_tg : 0 2014-01-07 18:55:47 2014-01-07 18:55:47 rt_rq[3]:/ 2014-01-07 18:55:47 .rt_nr_running : 0 2014-01-07 18:55:47 .rt_throttled : 0 2014-01-07 18:55:47 .rt_time : 0.000000 2014-01-07 18:55:47 .rt_runtime : 950.000000 2014-01-07 18:55:47 2014-01-07 18:55:47 runnable tasks: 2014-01-07 18:55:47 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:47 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:47 2014-01-07 18:55:47 cpu#4, 2593.668 MHz 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .nr_switches : 20409366 2014-01-07 18:55:47 .nr_load_updates : 987062 2014-01-07 18:55:47 .nr_uninterruptible : 0 2014-01-07 18:55:47 .next_balance : 4300.394715 2014-01-07 18:55:47 .curr->pid : 0 2014-01-07 18:55:47 .clock : 5717188.129744 2014-01-07 18:55:47 .cpu_load[0] : 0 2014-01-07 18:55:47 .cpu_load[1] : 0 2014-01-07 18:55:47 .cpu_load[2] : 0 2014-01-07 18:55:47 .cpu_load[3] : 0 2014-01-07 18:55:47 .cpu_load[4] : 0 2014-01-07 18:55:47 .yld_count : 0 2014-01-07 18:55:47 .sched_switch : 0 2014-01-07 18:55:47 .sched_count : 21575966 2014-01-07 18:55:47 .sched_goidle : 9717440 2014-01-07 18:55:47 .avg_idle : 1000000 2014-01-07 18:55:47 .ttwu_count : 10599510 2014-01-07 18:55:47 .ttwu_local : 1539756 2014-01-07 18:55:47 .bkl_count : 56 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[4]:/ 2014-01-07 18:55:47 .exec_clock : 478127.036229 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 7845447.025949 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -1415347.797069 2014-01-07 18:55:47 .nr_spread_over : 3641 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .load_avg : 0.000000 2014-01-07 18:55:47 .load_period : 0.000000 2014-01-07 18:55:47 .load_contrib : 0 2014-01-07 18:55:47 .load_tg : 0 2014-01-07 18:55:47 2014-01-07 18:55:47 rt_rq[4]:/ 2014-01-07 18:55:47 .rt_nr_running : 0 2014-01-07 18:55:47 .rt_throttled : 0 2014-01-07 18:55:47 .rt_time : 0.000000 2014-01-07 18:55:47 .rt_runtime : 950.000000 2014-01-07 18:55:47 2014-01-07 18:55:47 runnable tasks: 2014-01-07 18:55:47 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:47 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:47 2014-01-07 18:55:47 cpu#5, 2593.668 MHz 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .nr_switches : 18356292 2014-01-07 18:55:47 .nr_load_updates : 855913 2014-01-07 18:55:47 .nr_uninterruptible : 0 2014-01-07 18:55:47 .next_balance : 4300.395331 2014-01-07 18:55:47 .curr->pid : 0 2014-01-07 18:55:47 .clock : 5717376.678403 2014-01-07 18:55:47 .cpu_load[0] : 0 2014-01-07 18:55:47 .cpu_load[1] : 0 2014-01-07 18:55:47 .cpu_load[2] : 0 2014-01-07 18:55:47 .cpu_load[3] : 0 2014-01-07 18:55:47 .cpu_load[4] : 1 2014-01-07 18:55:47 .yld_count : 0 2014-01-07 18:55:47 .sched_switch : 0 2014-01-07 18:55:47 .sched_count : 19284689 2014-01-07 18:55:47 .sched_goidle : 8712458 2014-01-07 18:55:47 .avg_idle : 1000000 2014-01-07 18:55:47 .ttwu_count : 9551838 2014-01-07 18:55:47 .ttwu_local : 1667279 2014-01-07 18:55:47 .bkl_count : 163 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[5]:/ 2014-01-07 18:55:47 .exec_clock : 473893.678156 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 7884429.861337 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -1376364.961681 2014-01-07 18:55:47 .nr_spread_over : 4009 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .load_avg : 0.000000 2014-01-07 18:55:47 .load_period : 0.000000 2014-01-07 18:55:47 .load_contrib : 0 2014-01-07 18:55:47 .load_tg : 0 2014-01-07 18:55:47 2014-01-07 18:55:47 rt_rq[5]:/ 2014-01-07 18:55:47 .rt_nr_running : 0 2014-01-07 18:55:47 .rt_throttled : 0 2014-01-07 18:55:47 .rt_time : 0.000000 2014-01-07 18:55:47 .rt_runtime : 950.000000 2014-01-07 18:55:47 2014-01-07 18:55:47 runnable tasks: 2014-01-07 18:55:47 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:47 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:47 2014-01-07 18:55:47 cpu#6, 2593.668 MHz 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .nr_switches : 14983608 2014-01-07 18:55:47 .nr_load_updates : 894497 2014-01-07 18:55:47 .nr_uninterruptible : 0 2014-01-07 18:55:47 .next_balance : 4300.395141 2014-01-07 18:55:47 .curr->pid : 0 2014-01-07 18:55:47 .clock : 5717567.286208 2014-01-07 18:55:47 .cpu_load[0] : 0 2014-01-07 18:55:47 .cpu_load[1] : 2 2014-01-07 18:55:47 .cpu_load[2] : 12 2014-01-07 18:55:47 .cpu_load[3] : 34 2014-01-07 18:55:47 .cpu_load[4] : 60 2014-01-07 18:55:47 .yld_count : 0 2014-01-07 18:55:47 .sched_switch : 0 2014-01-07 18:55:47 .sched_count : 15576126 2014-01-07 18:55:47 .sched_goidle : 7099898 2014-01-07 18:55:47 .avg_idle : 1000000 2014-01-07 18:55:47 .ttwu_count : 7794764 2014-01-07 18:55:47 .ttwu_local : 1060360 2014-01-07 18:55:47 .bkl_count : 34 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[6]:/ 2014-01-07 18:55:47 .exec_clock : 394700.497000 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 8044750.931685 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -1216043.891333 2014-01-07 18:55:47 .nr_spread_over : 4362 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 1039 2014-01-07 18:55:47 .load_avg : 0.000000 2014-01-07 18:55:47 .load_period : 0.000000 2014-01-07 18:55:47 .load_contrib : 0 2014-01-07 18:55:47 .load_tg : 0 2014-01-07 18:55:47 2014-01-07 18:55:47 rt_rq[6]:/ 2014-01-07 18:55:47 .rt_nr_running : 0 2014-01-07 18:55:47 .rt_throttled : 0 2014-01-07 18:55:47 .rt_time : 0.000000 2014-01-07 18:55:47 .rt_runtime : 950.000000 2014-01-07 18:55:47 2014-01-07 18:55:47 runnable tasks: 2014-01-07 18:55:47 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:47 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:47 2014-01-07 18:55:47 cpu#7, 2593.668 MHz 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .nr_switches : 12468888 2014-01-07 18:55:47 .nr_load_updates : 845492 2014-01-07 18:55:47 .nr_uninterruptible : 0 2014-01-07 18:55:47 .next_balance : 4300.395694 2014-01-07 18:55:47 .curr->pid : 0 2014-01-07 18:55:47 .clock : 5717755.922973 2014-01-07 18:55:47 .cpu_load[0] : 0 2014-01-07 18:55:47 .cpu_load[1] : 0 2014-01-07 18:55:47 .cpu_load[2] : 0 2014-01-07 18:55:47 .cpu_load[3] : 0 2014-01-07 18:55:47 .cpu_load[4] : 0 2014-01-07 18:55:47 .yld_count : 0 2014-01-07 18:55:47 .sched_switch : 0 2014-01-07 18:55:47 .sched_count : 13093366 2014-01-07 18:55:47 .sched_goidle : 5878966 2014-01-07 18:55:47 .avg_idle : 1000000 2014-01-07 18:55:47 .ttwu_count : 6498062 2014-01-07 18:55:47 .ttwu_local : 1304695 2014-01-07 18:55:47 .bkl_count : 40 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[7]:/ 2014-01-07 18:55:47 .exec_clock : 331958.404089 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 8321000.355678 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -939794.467340 2014-01-07 18:55:47 .nr_spread_over : 4117 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .load_avg : 0.000000 2014-01-07 18:55:47 .load_period : 0.000000 2014-01-07 18:55:47 .load_contrib : 0 2014-01-07 18:55:47 .load_tg : 0 2014-01-07 18:55:47 2014-01-07 18:55:47 rt_rq[7]:/ 2014-01-07 18:55:47 .rt_nr_running : 0 2014-01-07 18:55:47 .rt_throttled : 0 2014-01-07 18:55:47 .rt_time : 0.000000 2014-01-07 18:55:47 .rt_runtime : 950.000000 2014-01-07 18:55:47 2014-01-07 18:55:47 runnable tasks: 2014-01-07 18:55:47 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:47 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:47 2014-01-07 18:55:47 cpu#8, 2593.668 MHz 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:47 .nr_switches : 14768063 2014-01-07 18:55:47 .nr_load_updates : 748867 2014-01-07 18:55:47 .nr_uninterruptible : 0 2014-01-07 18:55:47 .next_balance : 4300.395468 2014-01-07 18:55:47 .curr->pid : 0 2014-01-07 18:55:47 .clock : 5717944.810795 2014-01-07 18:55:47 .cpu_load[0] : 0 2014-01-07 18:55:47 .cpu_load[1] : 0 2014-01-07 18:55:47 .cpu_load[2] : 0 2014-01-07 18:55:47 .cpu_load[3] : 0 2014-01-07 18:55:47 .cpu_load[4] : 0 2014-01-07 18:55:47 .yld_count : 0 2014-01-07 18:55:47 .sched_switch : 0 2014-01-07 18:55:47 .sched_count : 17439023 2014-01-07 18:55:47 .sched_goidle : 6922958 2014-01-07 18:55:47 .avg_idle : 1000000 2014-01-07 18:55:47 .ttwu_count : 7747495 2014-01-07 18:55:47 .ttwu_local : 1122942 2014-01-07 18:55:47 .bkl_count : 6807 2014-01-07 18:55:47 2014-01-07 18:55:47 cfs_rq[8]:/ 2014-01-07 18:55:47 .exec_clock : 468756.454465 2014-01-07 18:55:47 .MIN_vruntime : 0.000001 2014-01-07 18:55:47 .min_vruntime : 8542944.814418 2014-01-07 18:55:47 .max_vruntime : 0.000001 2014-01-07 18:55:47 .spread : 0.000000 2014-01-07 18:55:47 .spread0 : -717850.008600 2014-01-07 18:55:47 .nr_spread_over : 4559 2014-01-07 18:55:47 .nr_running : 0 2014-01-07 18:55:47 .load : 0 2014-01-07 18:55:48 .load_avg : 0.000000 2014-01-07 18:55:48 .load_period : 0.000000 2014-01-07 18:55:48 .load_contrib : 0 2014-01-07 18:55:48 .load_tg : 0 2014-01-07 18:55:48 2014-01-07 18:55:48 rt_rq[8]:/ 2014-01-07 18:55:48 .rt_nr_running : 0 2014-01-07 18:55:48 .rt_throttled : 0 2014-01-07 18:55:48 .rt_time : 0.000000 2014-01-07 18:55:48 .rt_runtime : 950.000000 2014-01-07 18:55:48 2014-01-07 18:55:48 runnable tasks: 2014-01-07 18:55:48 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:48 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:48 R z_rd_int/8 6762 8543127.309036 698081 139 8543127.309036 44771.618729 5391925.247325 / 2014-01-07 18:55:48 2014-01-07 18:55:48 cpu#9, 2593.668 MHz 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .nr_switches : 14989919 2014-01-07 18:55:48 .nr_load_updates : 1158320 2014-01-07 18:55:48 .nr_uninterruptible : 0 2014-01-07 18:55:48 .next_balance : 4300.396024 2014-01-07 18:55:48 .curr->pid : 0 2014-01-07 18:55:48 .clock : 5718146.502410 2014-01-07 18:55:48 .cpu_load[0] : 0 2014-01-07 18:55:48 .cpu_load[1] : 0 2014-01-07 18:55:48 .cpu_load[2] : 0 2014-01-07 18:55:48 .cpu_load[3] : 99 2014-01-07 18:55:48 .cpu_load[4] : 553 2014-01-07 18:55:48 .yld_count : 0 2014-01-07 18:55:48 .sched_switch : 0 2014-01-07 18:55:48 .sched_count : 17857996 2014-01-07 18:55:48 .sched_goidle : 7083068 2014-01-07 18:55:48 .avg_idle : 861793 2014-01-07 18:55:48 .ttwu_count : 7827718 2014-01-07 18:55:48 .ttwu_local : 681559 2014-01-07 18:55:48 .bkl_count : 113 2014-01-07 18:55:48 2014-01-07 18:55:48 cfs_rq[9]:/ 2014-01-07 18:55:48 .exec_clock : 450052.566691 2014-01-07 18:55:48 .MIN_vruntime : 0.000001 2014-01-07 18:55:48 .min_vruntime : 9053106.677398 2014-01-07 18:55:48 .max_vruntime : 0.000001 2014-01-07 18:55:48 .spread : 0.000000 2014-01-07 18:55:48 .spread0 : -207688.145620 2014-01-07 18:55:48 .nr_spread_over : 4460 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .load_avg : 0.000000 2014-01-07 18:55:48 .load_period : 0.000000 2014-01-07 18:55:48 .load_contrib : 0 2014-01-07 18:55:48 .load_tg : 0 2014-01-07 18:55:48 2014-01-07 18:55:48 rt_rq[9]:/ 2014-01-07 18:55:48 .rt_nr_running : 0 2014-01-07 18:55:48 .rt_throttled : 0 2014-01-07 18:55:48 .rt_time : 0.000000 2014-01-07 18:55:48 .rt_runtime : 950.000000 2014-01-07 18:55:48 2014-01-07 18:55:48 runnable tasks: 2014-01-07 18:55:48 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:48 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:48 2014-01-07 18:55:48 cpu#10, 2593.668 MHz 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .nr_switches : 13953746 2014-01-07 18:55:48 .nr_load_updates : 886367 2014-01-07 18:55:48 .nr_uninterruptible : 0 2014-01-07 18:55:48 .next_balance : 4300.396208 2014-01-07 18:55:48 .curr->pid : 0 2014-01-07 18:55:48 .clock : 5718334.732869 2014-01-07 18:55:48 .cpu_load[0] : 0 2014-01-07 18:55:48 .cpu_load[1] : 0 2014-01-07 18:55:48 .cpu_load[2] : 0 2014-01-07 18:55:48 .cpu_load[3] : 0 2014-01-07 18:55:48 .cpu_load[4] : 0 2014-01-07 18:55:48 .yld_count : 0 2014-01-07 18:55:48 .sched_switch : 0 2014-01-07 18:55:48 .sched_count : 15257115 2014-01-07 18:55:48 .sched_goidle : 6605895 2014-01-07 18:55:48 .avg_idle : 964651 2014-01-07 18:55:48 .ttwu_count : 7245729 2014-01-07 18:55:48 .ttwu_local : 553441 2014-01-07 18:55:48 .bkl_count : 99 2014-01-07 18:55:48 2014-01-07 18:55:48 cfs_rq[10]:/ 2014-01-07 18:55:48 .exec_clock : 365384.079679 2014-01-07 18:55:48 .MIN_vruntime : 0.000001 2014-01-07 18:55:48 .min_vruntime : 8985483.044194 2014-01-07 18:55:48 .max_vruntime : 0.000001 2014-01-07 18:55:48 .spread : 0.000000 2014-01-07 18:55:48 .spread0 : -275311.778824 2014-01-07 18:55:48 .nr_spread_over : 4630 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .load_avg : 0.000000 2014-01-07 18:55:48 .load_period : 0.000000 2014-01-07 18:55:48 .load_contrib : 0 2014-01-07 18:55:48 .load_tg : 0 2014-01-07 18:55:48 2014-01-07 18:55:48 rt_rq[10]:/ 2014-01-07 18:55:48 .rt_nr_running : 0 2014-01-07 18:55:48 .rt_throttled : 0 2014-01-07 18:55:48 .rt_time : 0.000000 2014-01-07 18:55:48 .rt_runtime : 950.000000 2014-01-07 18:55:48 2014-01-07 18:55:48 runnable tasks: 2014-01-07 18:55:48 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:48 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:48 2014-01-07 18:55:48 cpu#11, 2593.668 MHz 2014-01-07 18:55:48 .nr_running : 3 2014-01-07 18:55:48 .load : 1039 2014-01-07 18:55:48 .nr_switches : 10890299 2014-01-07 18:55:48 .nr_load_updates : 735485 2014-01-07 18:55:48 .nr_uninterruptible : 0 2014-01-07 18:55:48 .next_balance : 4300.349194 2014-01-07 18:55:48 .curr->pid : 9359 2014-01-07 18:55:48 .clock : 5671632.465362 2014-01-07 18:55:48 .cpu_load[0] : 0 2014-01-07 18:55:48 .cpu_load[1] : 0 2014-01-07 18:55:48 .cpu_load[2] : 0 2014-01-07 18:55:48 .cpu_load[3] : 0 2014-01-07 18:55:48 .cpu_load[4] : 0 2014-01-07 18:55:48 .yld_count : 0 2014-01-07 18:55:48 .sched_switch : 0 2014-01-07 18:55:48 .sched_count : 11539153 2014-01-07 18:55:48 .sched_goidle : 5128012 2014-01-07 18:55:48 .avg_idle : 1000000 2014-01-07 18:55:48 .ttwu_count : 5663390 2014-01-07 18:55:48 .ttwu_local : 523110 2014-01-07 18:55:48 .bkl_count : 43 2014-01-07 18:55:48 2014-01-07 18:55:48 cfs_rq[11]:/ 2014-01-07 18:55:48 .exec_clock : 312013.852184 2014-01-07 18:55:48 .MIN_vruntime : 9232553.833536 2014-01-07 18:55:48 .min_vruntime : 9232553.833536 2014-01-07 18:55:48 .max_vruntime : 9232553.833536 2014-01-07 18:55:48 .spread : 0.000000 2014-01-07 18:55:48 .spread0 : -28240.989482 2014-01-07 18:55:48 .nr_spread_over : 4811 2014-01-07 18:55:48 .nr_running : 2 2014-01-07 18:55:48 .load : 1039 2014-01-07 18:55:48 .load_avg : 0.000000 2014-01-07 18:55:48 .load_period : 0.000000 2014-01-07 18:55:48 .load_contrib : 0 2014-01-07 18:55:48 .load_tg : 0 2014-01-07 18:55:48 2014-01-07 18:55:48 rt_rq[11]:/ 2014-01-07 18:55:48 .rt_nr_running : 1 2014-01-07 18:55:48 .rt_throttled : 0 2014-01-07 18:55:48 .rt_time : 0.000000 2014-01-07 18:55:48 .rt_runtime : 950.000000 2014-01-07 18:55:48 2014-01-07 18:55:48 runnable tasks: 2014-01-07 18:55:48 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:48 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:48 migration/11 47 776.930676 168 0 776.930676 4.990417 0.000000 / 2014-01-07 18:55:48 z_rd_int/11 6765 9232553.833536 682698 139 9232553.833536 52114.340061 5350745.665069 / 2014-01-07 18:55:48 R anacron 9359 9232544.384832 5 120 9232544.384832 2.617681 3233309.249931 / 2014-01-07 18:55:48 2014-01-07 18:55:48 cpu#12, 2593.668 MHz 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .nr_switches : 17104240 2014-01-07 18:55:48 .nr_load_updates : 923272 2014-01-07 18:55:48 .nr_uninterruptible : 0 2014-01-07 18:55:48 .next_balance : 4300.396310 2014-01-07 18:55:48 .curr->pid : 0 2014-01-07 18:55:48 .clock : 5718746.912674 2014-01-07 18:55:48 .cpu_load[0] : 0 2014-01-07 18:55:48 .cpu_load[1] : 0 2014-01-07 18:55:48 .cpu_load[2] : 2 2014-01-07 18:55:48 .cpu_load[3] : 17 2014-01-07 18:55:48 .cpu_load[4] : 42 2014-01-07 18:55:48 .yld_count : 0 2014-01-07 18:55:48 .sched_switch : 0 2014-01-07 18:55:48 .sched_count : 17899273 2014-01-07 18:55:48 .sched_goidle : 8063907 2014-01-07 18:55:48 .avg_idle : 738317 2014-01-07 18:55:48 .ttwu_count : 8940555 2014-01-07 18:55:48 .ttwu_local : 567989 2014-01-07 18:55:48 .bkl_count : 1 2014-01-07 18:55:48 2014-01-07 18:55:48 cfs_rq[12]:/ 2014-01-07 18:55:48 .exec_clock : 498165.481429 2014-01-07 18:55:48 .MIN_vruntime : 0.000001 2014-01-07 18:55:48 .min_vruntime : 8630969.773283 2014-01-07 18:55:48 .max_vruntime : 0.000001 2014-01-07 18:55:48 .spread : 0.000000 2014-01-07 18:55:48 .spread0 : -629825.049735 2014-01-07 18:55:48 .nr_spread_over : 4305 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .load_avg : 0.000000 2014-01-07 18:55:48 .load_period : 0.000000 2014-01-07 18:55:48 .load_contrib : 0 2014-01-07 18:55:48 .load_tg : 0 2014-01-07 18:55:48 2014-01-07 18:55:48 rt_rq[12]:/ 2014-01-07 18:55:48 .rt_nr_running : 0 2014-01-07 18:55:48 .rt_throttled : 0 2014-01-07 18:55:48 .rt_time : 0.000000 2014-01-07 18:55:48 .rt_runtime : 950.000000 2014-01-07 18:55:48 2014-01-07 18:55:48 runnable tasks: 2014-01-07 18:55:48 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:48 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:48 2014-01-07 18:55:48 cpu#13, 2593.668 MHz 2014-01-07 18:55:48 .nr_running : 1 2014-01-07 18:55:48 .load : 0 2014-01-07 18:55:48 .nr_switches : 16770225 2014-01-07 18:55:48 .nr_load_updates : 861575 2014-01-07 18:55:48 .nr_uninterruptible : 1 2014-01-07 18:55:48 .next_balance : 4300.396561 2014-01-07 18:55:48 .curr->pid : 0 2014-01-07 18:55:48 .clock : 5718937.558280 2014-01-07 18:55:48 .cpu_load[0] : 1024 2014-01-07 18:55:48 .cpu_load[1] : 281 2014-01-07 18:55:48 .cpu_load[2] : 367 2014-01-07 18:55:48 .cpu_load[3] : 156 2014-01-07 18:55:48 .cpu_load[4] : 150 2014-01-07 18:55:48 .yld_count : 0 2014-01-07 18:55:48 .sched_switch : 0 2014-01-07 18:55:48 .sched_count : 17437981 2014-01-07 18:55:48 .sched_goidle : 7917303 2014-01-07 18:55:48 .avg_idle : 1000000 2014-01-07 18:55:48 .ttwu_count : 8749265 2014-01-07 18:55:48 .ttwu_local : 561078 2014-01-07 18:55:48 .bkl_count : 5 2014-01-07 18:55:48 2014-01-07 18:55:48 cfs_rq[13]:/ 2014-01-07 18:55:48 .exec_clock : 466325.900308 2014-01-07 18:55:48 .MIN_vruntime : 0.000001 2014-01-07 18:55:48 .min_vruntime : 8667622.455924 2014-01-07 18:55:48 .max_vruntime : 0.000001 2014-01-07 18:55:48 .spread : 0.000000 2014-01-07 18:55:48 .spread0 : -593172.367094 2014-01-07 18:55:48 .nr_spread_over : 4343 2014-01-07 18:55:48 .nr_running : 0 2014-01-07 18:55:49 .load : 0 2014-01-07 18:55:49 .load_avg : 0.000000 2014-01-07 18:55:49 .load_period : 0.000000 2014-01-07 18:55:49 .load_contrib : 0 2014-01-07 18:55:49 .load_tg : 0 2014-01-07 18:55:49 2014-01-07 18:55:49 rt_rq[13]:/ 2014-01-07 18:55:49 .rt_nr_running : 0 2014-01-07 18:55:49 .rt_throttled : 0 2014-01-07 18:55:49 .rt_time : 0.000000 2014-01-07 18:55:49 .rt_runtime : 950.000000 2014-01-07 18:55:49 2014-01-07 18:55:49 runnable tasks: 2014-01-07 18:55:49 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:49 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:49 2014-01-07 18:55:49 cpu#14, 2593.668 MHz 2014-01-07 18:55:49 .nr_running : 0 2014-01-07 18:55:49 .load : 0 2014-01-07 18:55:49 .nr_switches : 15689058 2014-01-07 18:55:49 .nr_load_updates : 791830 2014-01-07 18:55:49 .nr_uninterruptible : 0 2014-01-07 18:55:49 .next_balance : 4300.396699 2014-01-07 18:55:49 .curr->pid : 0 2014-01-07 18:55:49 .clock : 5719122.325500 2014-01-07 18:55:49 .cpu_load[0] : 1024 2014-01-07 18:55:49 .cpu_load[1] : 512 2014-01-07 18:55:49 .cpu_load[2] : 256 2014-01-07 18:55:49 .cpu_load[3] : 1 2014-01-07 18:55:49 .cpu_load[4] : 9 2014-01-07 18:55:49 .yld_count : 0 2014-01-07 18:55:49 .sched_switch : 0 2014-01-07 18:55:49 .sched_count : 16232095 2014-01-07 18:55:49 .sched_goidle : 7437609 2014-01-07 18:55:49 .avg_idle : 687878 2014-01-07 18:55:49 .ttwu_count : 8143881 2014-01-07 18:55:49 .ttwu_local : 547051 2014-01-07 18:55:49 .bkl_count : 2 2014-01-07 18:55:49 2014-01-07 18:55:49 cfs_rq[14]:/ 2014-01-07 18:55:49 .exec_clock : 387641.868477 2014-01-07 18:55:49 .MIN_vruntime : 0.000001 2014-01-07 18:55:49 .min_vruntime : 8855775.586625 2014-01-07 18:55:49 .max_vruntime : 0.000001 2014-01-07 18:55:49 .spread : 0.000000 2014-01-07 18:55:49 .spread0 : -405019.236393 2014-01-07 18:55:49 .nr_spread_over : 4393 2014-01-07 18:55:49 .nr_running : 0 2014-01-07 18:55:49 .load : 0 2014-01-07 18:55:49 .load_avg : 0.000000 2014-01-07 18:55:49 .load_period : 0.000000 2014-01-07 18:55:49 .load_contrib : 0 2014-01-07 18:55:49 .load_tg : 0 2014-01-07 18:55:49 2014-01-07 18:55:49 rt_rq[14]:/ 2014-01-07 18:55:49 .rt_nr_running : 0 2014-01-07 18:55:49 .rt_throttled : 0 2014-01-07 18:55:49 .rt_time : 0.000000 2014-01-07 18:55:49 .rt_runtime : 950.000000 2014-01-07 18:55:49 2014-01-07 18:55:49 runnable tasks: 2014-01-07 18:55:49 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:49 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:49 2014-01-07 18:55:49 cpu#15, 2593.668 MHz 2014-01-07 18:55:49 .nr_running : 0 2014-01-07 18:55:49 .load : 0 2014-01-07 18:55:49 .nr_switches : 12361791 2014-01-07 18:55:49 .nr_load_updates : 744799 2014-01-07 18:55:49 .nr_uninterruptible : 0 2014-01-07 18:55:49 .next_balance : 4300.397273 2014-01-07 18:55:49 .curr->pid : 0 2014-01-07 18:55:49 .clock : 5719315.651247 2014-01-07 18:55:49 .cpu_load[0] : 15 2014-01-07 18:55:49 .cpu_load[1] : 8 2014-01-07 18:55:49 .cpu_load[2] : 0 2014-01-07 18:55:49 .cpu_load[3] : 0 2014-01-07 18:55:49 .cpu_load[4] : 0 2014-01-07 18:55:49 .yld_count : 0 2014-01-07 18:55:49 .sched_switch : 0 2014-01-07 18:55:49 .sched_count : 12832396 2014-01-07 18:55:49 .sched_goidle : 5836562 2014-01-07 18:55:49 .avg_idle : 1000000 2014-01-07 18:55:49 .ttwu_count : 6424056 2014-01-07 18:55:49 .ttwu_local : 528988 2014-01-07 18:55:49 .bkl_count : 34 2014-01-07 18:55:49 2014-01-07 18:55:49 cfs_rq[15]:/ 2014-01-07 18:55:49 .exec_clock : 332089.030819 2014-01-07 18:55:49 .MIN_vruntime : 0.000001 2014-01-07 18:55:49 .min_vruntime : 9243140.994115 2014-01-07 18:55:49 .max_vruntime : 0.000001 2014-01-07 18:55:49 .spread : 0.000000 2014-01-07 18:55:49 .spread0 : -17653.828903 2014-01-07 18:55:49 .nr_spread_over : 4544 2014-01-07 18:55:49 .nr_running : 0 2014-01-07 18:55:49 .load : 0 2014-01-07 18:55:49 .load_avg : 0.000000 2014-01-07 18:55:49 .load_period : 0.000000 2014-01-07 18:55:49 .load_contrib : 0 2014-01-07 18:55:49 .load_tg : 0 2014-01-07 18:55:49 2014-01-07 18:55:49 rt_rq[15]:/ 2014-01-07 18:55:49 .rt_nr_running : 0 2014-01-07 18:55:49 .rt_throttled : 0 2014-01-07 18:55:49 .rt_time : 0.000000 2014-01-07 18:55:49 .rt_runtime : 950.000000 2014-01-07 18:55:49 2014-01-07 18:55:49 runnable tasks: 2014-01-07 18:55:49 task PID tree-key switches prio exec-runtime sum-exec sum-sleep 2014-01-07 18:55:49 ---------------------------------------------------------------------------------------------------------- 2014-01-07 18:55:49 z_rd_int/15 6769 9243219.100260 694708 139 9243221.988350 51344.485076 5398936.125468 / 2014-01-07 18:55:49 2014-01-07 18:55:49 LNetError: 5621:0:(o2iblnd_cb.c:3012:kiblnd_check_txs_locked()) Timed out tx: active_txs, 42 seconds 2014-01-07 18:55:49 LNetError: 5621:0:(o2iblnd_cb.c:3075:kiblnd_check_conns()) Timed out RDMA with 192.168.123.245@o2ib1 (0): c: 0, oc: 0, rc: 2 2014-01-07 18:55:49 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c4b2000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b882c000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd4fe000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbd86000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b828a000 2014-01-07 18:55:49 LustreError: 5621:0:(events.c:450:server_bulk_callback()) event type 5, status -103, desc ffff88007f92e000 2014-01-07 18:55:49 LustreError: 5621:0:(events.c:450:server_bulk_callback()) event type 5, status -103, desc ffff88007fab0000 2014-01-07 18:55:49 LustreError: 5621:0:(events.c:450:server_bulk_callback()) event type 5, status -103, desc ffff8800baaa8000 2014-01-07 18:55:49 LustreError: 5621:0:(events.c:450:server_bulk_callback()) event type 5, status -103, desc ffff8800bd640000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fb20000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bcfa6000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f9de000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f048000 2014-01-07 18:55:49 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc486000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd2a0000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f3da000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fb92000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd026000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fb32000 2014-01-07 18:55:49 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f3e4000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd588000 2014-01-07 18:55:49 Lustre: lustre-OST0008: Client e8989ae7-b394-ab3e-ba0f-adcbd57a4ec7 (at 192.168.124.20@o2ib1) reconnecting 2014-01-07 18:55:49 Lustre: lustre-OST0008: Client e8989ae7-b394-ab3e-ba0f-adcbd57a4ec7 (at 192.168.124.20@o2ib1) refused reconnection, still busy with 9 active RPCs 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f24a000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd528000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd244000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b8760000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd1d4000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fbd4000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f050000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbe62000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd78e000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc69c000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880782092500 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bb874000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc03e000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fab8000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bb954000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc362000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f3fe000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc43e000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b8926000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fcaa000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd028000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd5a4000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbd3a000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc378000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd264000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b7f7a000 2014-01-07 18:55:49 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f2ae000 2014-01-07 18:55:49 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bcbce000 2014-01-07 18:55:49 LustreError: 7434:0:(ldlm_lib.c:2707:target_bulk_io()) @@@ Reconnect on bulk PUT req@ffff8800bd552c00 x1456627275602504/t0(0) o3->e8989ae7-b394-ab3e-ba0f-adcbd57a4ec7@192.168.124.20@o2ib1:0/0 lens 488/432 e 1 to 0 dl 1389149885 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:50 Lustre: lustre-OST0008: Bulk IO read error with e8989ae7-b394-ab3e-ba0f-adcbd57a4ec7 (at 192.168.124.20@o2ib1), client will retry: rc -110 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd616000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbd32000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f3f8000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800ba288000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f088000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f300000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbd3c000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd354000 2014-01-07 18:55:50 Lustre: lustre-OST0008: Client b9b461b0-4ad3-be8f-c558-e9949458d640 (at 192.168.123.245@o2ib1) reconnecting 2014-01-07 18:55:50 Lustre: lustre-OST0008: Client b9b461b0-4ad3-be8f-c558-e9949458d640 (at 192.168.123.245@o2ib1) refused reconnection, still busy with 9 active RPCs 2014-01-07 18:55:50 LustreError: 5624:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f32a000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b8916000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbd76000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b828c000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f2a2000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bcfb6000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fbae000 2014-01-07 18:55:50 LustreError: 5625:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd662000 2014-01-07 18:55:50 LustreError: 7471:0:(ldlm_lib.c:2707:target_bulk_io()) @@@ Reconnect on bulk PUT req@ffff8800bd670000 x1456627274408108/t0(0) o3->b9b461b0-4ad3-be8f-c558-e9949458d640@192.168.123.245@o2ib1:0/0 lens 488/432 e 1 to 0 dl 1389149886 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:50 Lustre: lustre-OST0008: Bulk IO read error with b9b461b0-4ad3-be8f-c558-e9949458d640 (at 192.168.123.245@o2ib1), client will retry: rc -110 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f8fc000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800b9886000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007fcb4000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bbfd0000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd1fa000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd478000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd1b8000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd74a000 2014-01-07 18:55:50 LustreError: 5623:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd002000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd2d2000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bcf94000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd572000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd234000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff88007f372000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bc6d6000 2014-01-07 18:55:50 LustreError: 5622:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff8800bd4b8000 2014-01-07 18:55:50 LustreError: 7467:0:(ldlm_lib.c:2707:target_bulk_io()) @@@ Reconnect on bulk PUT req@ffff8800bbcdd800 x1456627275495756/t0(0) o3->e5340887-a2f3-1b67-ee4f-f16f2c93ee62@192.168.124.5@o2ib1:0/0 lens 488/432 e 1 to 0 dl 1389149874 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:50 LustreError: 7467:0:(ldlm_lib.c:2707:target_bulk_io()) Skipped 40 previous similar messages 2014-01-07 18:55:50 Lustre: lustre-OST0008: Bulk IO read error with e5340887-a2f3-1b67-ee4f-f16f2c93ee62 (at 192.168.124.5@o2ib1), client will retry: rc -110 2014-01-07 18:55:50 Lustre: Skipped 40 previous similar messages 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7adfc000 2014-01-07 18:55:50 Lustre: lustre-OST0008: Client 8a70167d-eac7-67f7-3b07-5dda0691a8e3 (at 192.168.124.24@o2ib1) reconnecting 2014-01-07 18:55:50 Lustre: lustre-OST0008: Client d2d09131-bbb0-06c6-9382-b0d52b88fe27 (at 192.168.124.27@o2ib1) refused reconnection, still busy with 9 active RPCs 2014-01-07 18:55:50 Lustre: Skipped 8 previous similar messages 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7afee000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7be68000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c238000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c6b0000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c692000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7bfb2000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7d730000 2014-01-07 18:55:50 LNet: 5629:0:(o2iblnd_cb.c:381:kiblnd_handle_rx()) PUT_NACK from 192.168.124.19@o2ib1 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7aef4000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a8c4000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c408000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c1d6000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7bfa0000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c498000 2014-01-07 18:55:50 LustreError: 7129:0:(ldlm_lib.c:2726:target_bulk_io()) @@@ bulk PUT failed: rc -107 req@ffff8800bc854000 x1456627275490960/t0(0) o3->6c947908-c970-3c62-fece-8415e8e853bc@192.168.124.13@o2ib1:0/0 lens 488/432 e 0 to 0 dl 1389149861 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c694000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7af02000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7bf94000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c736000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c540000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c730000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c7d8000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7aea6000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a84e000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7af22000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7af70000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c7ce000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7ae7a000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7bdf2000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7ad08000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7af1e000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7ba4e000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c616000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c676000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a870000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a86a000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a820000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c79e000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a868000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a800000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a8e0000 2014-01-07 18:55:50 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7af96000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7afa2000 2014-01-07 18:55:50 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7c086000 2014-01-07 18:55:51 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7ae5e000 2014-01-07 18:55:51 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7afde000 2014-01-07 18:55:51 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a8c0000 2014-01-07 18:55:51 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7ad02000 2014-01-07 18:55:51 LustreError: 5629:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a8ac000 2014-01-07 18:55:51 LustreError: 5628:0:(events.c:450:server_bulk_callback()) event type 5, status -5, desc ffff880e7a806000 2014-01-07 18:55:51 LustreError: 7423:0:(ldlm_lib.c:2726:target_bulk_io()) @@@ bulk PUT failed: rc -107 req@ffff880e7f54f400 x1456627275717436/t0(0) o3->d2d09131-bbb0-06c6-9382-b0d52b88fe27@192.168.124.27@o2ib1:0/0 lens 488/432 e 0 to 0 dl 1389149817 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:51 Lustre: Skipped 14 previous similar messages 2014-01-07 18:55:51 Lustre: lustre-OST0008: Bulk IO read error with 169b44e5-280f-7df8-e1f7-e95385d34cd2 (at 192.168.124.11@o2ib1), client will retry: rc -107 2014-01-07 18:55:51 Lustre: Skipped 76 previous similar messages 2014-01-07 18:55:51 LustreError: 7350:0:(ldlm_lib.c:2707:target_bulk_io()) @@@ Reconnect on bulk PUT req@ffff880e7d7e9c00 x1456627275717104/t0(0) o3->d2d09131-bbb0-06c6-9382-b0d52b88fe27@192.168.124.27@o2ib1:0/0 lens 488/432 e 1 to 0 dl 1389149797 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:51 LustreError: 7350:0:(ldlm_lib.c:2707:target_bulk_io()) Skipped 97 previous similar messages 2014-01-07 18:55:51 LustreError: 7138:0:(ldlm_lib.c:2726:target_bulk_io()) @@@ bulk PUT failed: rc -107 req@ffff880e7d671800 x1456627275453636/t0(0) o3->8a70167d-eac7-67f7-3b07-5dda0691a8e3@192.168.124.24@o2ib1:0/0 lens 488/432 e 0 to 0 dl 1389149817 ref 1 fl Interpret:/0/0 rc 0/0 2014-01-07 18:55:51 LustreError: 7138:0:(ldlm_lib.c:2726:target_bulk_io()) Skipped 1 previous similar message 2014-01-07 18:55:52 Lustre: lustre-OST0008: Client cdfba5c2-f223-aefa-92ea-9a714c1a35d1 (at 192.168.124.36@o2ib1) reconnecting 2014-01-07 18:55:52 Lustre: Skipped 3 previous similar messages 2014-01-07 18:56:17 Lustre: lustre-OST0008: Client b9b461b0-4ad3-be8f-c558-e9949458d640 (at 192.168.123.245@o2ib1) reconnecting 2014-01-07 18:56:17 Lustre: Skipped 12 previous similar messages