May 17 04:02:04 ALPL202 syslogd 1.4.1: restart. May 17 04:22:01 ALPL202 kernel: md: syncing RAID array md0 May 17 04:22:01 ALPL202 kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. May 17 04:22:01 ALPL202 kernel: md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. May 17 04:22:01 ALPL202 kernel: md: using 128k window, over a total of 795072 blocks. May 17 04:22:01 ALPL202 kernel: md: delaying resync of md2 until md0 has finished resync (they share one or more physical units) May 17 04:22:10 ALPL202 kernel: md: md0: sync done. May 17 04:22:11 ALPL202 kernel: md: syncing RAID array md2 May 17 04:22:11 ALPL202 kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. May 17 04:22:11 ALPL202 kernel: md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for reconstruction. May 17 04:22:11 ALPL202 kernel: md: using 128k window, over a total of 4192896 blocks. May 17 04:22:11 ALPL202 kernel: RAID1 conf printout: May 17 04:22:11 ALPL202 kernel: --- wd:2 rd:2 May 17 04:22:11 ALPL202 kernel: disk 0, wo:0, o:1, dev:sda1 May 17 04:22:11 ALPL202 kernel: disk 1, wo:0, o:1, dev:sdb1 May 17 04:23:40 ALPL202 kernel: md: md2: sync done. May 17 04:23:40 ALPL202 kernel: RAID1 conf printout: May 17 04:23:40 ALPL202 kernel: --- wd:2 rd:2 May 17 04:23:40 ALPL202 kernel: disk 0, wo:0, o:1, dev:sda3 May 17 04:23:40 ALPL202 kernel: disk 1, wo:0, o:1, dev:sdb3 May 17 04:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 05:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 06:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 07:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 08:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 09:44:47 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 10:44:48 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 11:44:48 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 11:57:05 ALPL202 kernel: INFO: task jbd2/dm-5-8:6346 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: jbd2/dm-5-8 D ffff8106550fa7a0 0 6346 427 6347 6345 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff810b74963720 0000000000000046 0000000000000050 ffff810640002c00 May 17 11:57:05 ALPL202 kernel: ffff81061d9507a0 000000000000000a ffff81061d9507a0 ffff8106553247a0 May 17 11:57:05 ALPL202 kernel: 00129c4266c2a5df 0000000000000324 ffff81061d950988 0000000700000010 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] alternate_node_alloc+0x70/0x8c May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_dquot_drop+0x2e/0x70 May 17 11:57:05 ALPL202 kernel: [] clear_inode+0xba/0x129 May 17 11:57:05 ALPL202 kernel: [] dispose_list+0x41/0xe0 May 17 11:57:05 ALPL202 kernel: [] shrink_icache_memory+0x1b7/0x1e7 May 17 11:57:05 ALPL202 kernel: [] shrink_slab+0xdc/0x153 May 17 11:57:05 ALPL202 kernel: [] zone_reclaim+0x235/0x2cd May 17 11:57:05 ALPL202 kernel: [] :ib_srp:srp_queuecommand+0x8ae/0x8c6 May 17 11:57:05 ALPL202 kernel: [] get_page_from_freelist+0xbf/0x442 May 17 11:57:05 ALPL202 kernel: [] try_to_wake_up+0x472/0x484 May 17 11:57:05 ALPL202 kernel: [] __alloc_pages+0x78/0x308 May 17 11:57:05 ALPL202 kernel: [] cache_grow+0x139/0x3c7 May 17 11:57:05 ALPL202 kernel: [] cache_alloc_refill+0x138/0x188 May 17 11:57:05 ALPL202 kernel: [] kmem_cache_alloc+0x6c/0x76 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:llcd_get+0xb4/0x480 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:llog_obd_repl_cancel+0x442/0x762 May 17 11:57:05 ALPL202 kernel: [] :obdclass:llog_cancel+0x251/0x2f0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:__llog_ctxt_put+0x26/0x150 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_cancel_cookies_cb+0x41b/0x630 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_cb_func+0x1b/0x130 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_commit_transaction+0xbb8/0x1120 May 17 11:57:05 ALPL202 kernel: [] lock_timer_base+0x1b/0x3c May 17 11:57:05 ALPL202 kernel: [] :jbd2:kjournald2+0x9a/0x1ec May 17 11:57:05 ALPL202 kernel: [] autoremove_wake_function+0x0/0x2e May 17 11:57:05 ALPL202 kernel: [] :jbd2:kjournald2+0x0/0x1ec May 17 11:57:05 ALPL202 kernel: [] keventd_create_kthread+0x0/0xc4 May 17 11:57:05 ALPL202 kernel: [] kthread+0xfe/0x132 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] keventd_create_kthread+0x0/0xc4 May 17 11:57:05 ALPL202 kernel: [] kthread+0x0/0x132 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_11:6550 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_11 D ffff81000100caa0 0 6550 1 6551 6548 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff8101f7c2b6d0 0000000000000046 ffff81083a676870 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff8102f7ea10c0 ffff810640047820 May 17 11:57:05 ALPL202 kernel: 00129c43926eb9b9 0000000000005162 ffff8102f7ea12a8 0000000110f8d830 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :lnet:lnet_ni_send+0x96/0xe0 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:target_committed_to_req+0x40/0x120 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_38:6577 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_38 D ffff8106550f2120 0 6577 1 6578 6576 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff81049bb296d0 0000000000000046 ffff810581c6bc28 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff8104305587a0 ffff8106552de860 May 17 11:57:05 ALPL202 kernel: 00129c415f98f498 0000000000005e31 ffff810430558988 0000000610f8d3e0 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_53:6592 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_53 D ffff8106550fa7a0 0 6592 1 6593 6591 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff81062b8256d0 0000000000000046 ffff81061f37bca8 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff81049ffb6100 ffff8106553247a0 May 17 11:57:05 ALPL202 kernel: 00129c429600fe53 00000000000050f7 ffff81049ffb62e8 0000000710f8d3e0 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_63:6602 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_63 D ffff8105d1156040 0 6602 1 6603 6601 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff8105f6acf8b0 0000000000000046 ffff810b1e2cf600 ffff8108e146ce40 May 17 11:57:05 ALPL202 kernel: 0000000000000001 000000000000000a ffff81049bb20040 ffff8105d1156040 May 17 11:57:05 ALPL202 kernel: 00129c413d572d11 000000000001ef87 ffff81049bb20228 00000006ffffffff May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_lock_remove_from_lru+0x74/0xe0 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_start+0x4c2/0x590 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_version_get_check+0x91/0x2a0 May 17 11:57:05 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:57:05 ALPL202 kernel: [] :libcfs:cfs_alloc+0x68/0xc0 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_destroy+0xa2d/0x1f90 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_blocking_ast+0x0/0x2a0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_completion_ast+0x0/0x880 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_add_version+0x34/0x110 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_pack_reply_flags+0x86a/0x950 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_resource_putref_internal+0x3ab/0x460 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_lock_put+0x372/0x3d0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_pack_reply+0x29/0xb0 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_destroy+0x660/0x790 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_get_opc+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x1566/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_87:6626 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_87 D ffff8106550e1420 0 6626 1 6627 6625 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff8103754e36d0 0000000000000046 ffff810376817a70 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff81061cb05040 ffff8106552227e0 May 17 11:57:05 ALPL202 kernel: 00129c4269f0ad92 0000000000005335 ffff81061cb05228 0000000410f8cb40 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_92:6631 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_92 D ffff8106550e1420 0 6631 1 6632 6630 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff81035d4916d0 0000000000000046 ffff81047d00bde8 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff81061cb0b860 ffff8106552227e0 May 17 11:57:05 ALPL202 kernel: 00129c416de78521 0000000000004fcc ffff81061cb0ba48 0000000410f8d830 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_100:6639 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_100 D ffff8106550e9aa0 0 6639 1 6640 6638 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff8105e208f8b0 0000000000000046 ffff810bed939600 ffff8106d8a43200 May 17 11:57:05 ALPL202 kernel: 0000000000000001 000000000000000a ffff8103759cc860 ffff81065527e820 May 17 11:57:05 ALPL202 kernel: 00129c413e90be13 000000000000a09a ffff8103759cca48 00000005ffffffff May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_lock_remove_from_lru+0x74/0xe0 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_start+0x4c2/0x590 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_version_get_check+0x91/0x2a0 May 17 11:57:05 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:57:05 ALPL202 kernel: [] :libcfs:cfs_alloc+0x68/0xc0 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_destroy+0xa2d/0x1f90 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_blocking_ast+0x0/0x2a0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_completion_ast+0x0/0x880 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_add_version+0x34/0x110 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_pack_reply_flags+0x86a/0x950 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_resource_putref_internal+0x3ab/0x460 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ldlm_lock_put+0x372/0x3d0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_pack_reply+0x29/0xb0 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_destroy+0x660/0x790 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_get_opc+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x1566/0x55c0 May 17 11:57:05 ALPL202 kernel: [] dequeue_task+0x18/0x37 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_108:6647 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_108 D ffff8106550e9aa0 0 6647 1 6648 6646 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff8105e51676d0 0000000000000046 ffff8102aae457f0 0000000000000000 May 17 11:57:05 ALPL202 kernel: 0000000000000000 000000000000000a ffff8101b8e04860 ffff81065527e820 May 17 11:57:05 ALPL202 kernel: 00129c416a756099 0000000000004a0b ffff8101b8e04a48 0000000510f8c7a8 May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:57:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:57:05 ALPL202 kernel: [] :jbd2:__jbd2_log_wait_for_space+0x5d/0x1ad May 17 11:57:05 ALPL202 kernel: [] :jbd2:start_this_handle+0x346/0x3cf May 17 11:57:05 ALPL202 kernel: [] :ldiskfs:ldiskfs_get_blocks+0xaa/0x210 May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_journal_start+0xad/0xe1 May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_brw_start+0x35c/0x490 May 17 11:57:05 ALPL202 kernel: [] :lquota:filter_quota_acquire+0x0/0x120 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0xe4d/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:57:05 ALPL202 kernel: INFO: task ll_ost_io_144:20839 blocked for more than 120 seconds. May 17 11:57:05 ALPL202 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. May 17 11:57:05 ALPL202 kernel: ll_ost_io_144 D ffff810c3e08f7a0 0 20839 1 20840 20838 (L-TLB) May 17 11:57:05 ALPL202 kernel: ffff810911f63800 0000000000000046 ffff81034d1e1460 ffff8105ee5276c0 May 17 11:57:05 ALPL202 kernel: ffff8105ee5276c0 000000000000000a ffff8108d1fee7e0 ffff810c3e08f7a0 May 17 11:57:05 ALPL202 kernel: 00129c413d2f65fb 000000000071882c ffff8108d1fee9c8 000000035a5a5a5a May 17 11:57:05 ALPL202 kernel: Call Trace: May 17 11:57:05 ALPL202 kernel: [] __wake_up+0x38/0x4f May 17 11:57:05 ALPL202 kernel: [] :jbd2:jbd2_log_wait_commit+0xa3/0xf5 May 17 11:57:05 ALPL202 kernel: [] autoremove_wake_function+0x0/0x2e May 17 11:57:05 ALPL202 kernel: [] :fsfilt_ldiskfs:fsfilt_ldiskfs_commit_wait+0xab/0xd0 May 17 11:57:05 ALPL202 kernel: [] :obdfilter:filter_commitrw_write+0x1eb4/0x2e70 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_brw_write+0x1c99/0x2480 May 17 11:57:05 ALPL202 kernel: [] :lnet:LNetMDBind+0x301/0x450 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_set_limit+0x35/0xf0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_send_reply+0x5e8/0x600 May 17 11:57:05 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lustre_msg_check_version_v2+0x8/0x20 May 17 11:57:05 ALPL202 kernel: [] :ost:ost_handle+0x2bc2/0x55c0 May 17 11:57:05 ALPL202 kernel: [] :obdclass:class_handle2object+0xe0/0x170 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:lock_res_and_lock+0xba/0xd0 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:__ldlm_handle2lock+0x2f8/0x360 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_server_handle_request+0x984/0xe00 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_wait_event+0x2e5/0x310 May 17 11:57:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0xf16/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:57:05 ALPL202 kernel: [] :ptlrpc:ptlrpc_main+0x0/0x10e0 May 17 11:57:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:57:05 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: Lustre: Service thread pid 20839 was inactive for 200.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: May 17 11:58:03 ALPL202 kernel: Pid: 20839, comm: ll_ost_io_144 May 17 11:58:03 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: Call Trace: May 17 11:58:03 ALPL202 kernel: [] __wake_up+0x38/0x4f May 17 11:58:03 ALPL202 kernel: [] jbd2_log_wait_commit+0xa3/0xf5 [jbd2] May 17 11:58:03 ALPL202 kernel: [] autoremove_wake_function+0x0/0x2e May 17 11:58:03 ALPL202 kernel: [] fsfilt_ldiskfs_commit_wait+0xab/0xd0 [fsfilt_ldiskfs] May 17 11:58:03 ALPL202 kernel: [] filter_commitrw_write+0x1eb4/0x2e70 [obdfilter] May 17 11:58:03 ALPL202 kernel: [] ost_brw_write+0x1c99/0x2480 [ost] May 17 11:58:03 ALPL202 kernel: [] LNetMDBind+0x301/0x450 [lnet] May 17 11:58:03 ALPL202 kernel: [] lustre_msg_set_limit+0x35/0xf0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ptlrpc_send_reply+0x5e8/0x600 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:58:03 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ost_handle+0x2bc2/0x55c0 [ost] May 17 11:58:03 ALPL202 kernel: [] class_handle2object+0xe0/0x170 [obdclass] May 17 11:58:03 ALPL202 kernel: [] lock_res_and_lock+0xba/0xd0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] __ldlm_handle2lock+0x2f8/0x360 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:58:03 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:58:03 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:58:03 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.20839 May 17 11:58:03 ALPL202 kernel: Lustre: Service thread pid 22745 was inactive for 200.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: May 17 11:58:03 ALPL202 kernel: Pid: 22745, comm: ll_ost_io_165 May 17 11:58:03 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: Call Trace: May 17 11:58:03 ALPL202 kernel: [] jbd2_log_wait_commit+0xa3/0xf5 [jbd2] May 17 11:58:03 ALPL202 kernel: [] autoremove_wake_function+0x0/0x2e May 17 11:58:03 ALPL202 kernel: [] jbd2_log_do_checkpoint+0x160/0x48f [jbd2] May 17 11:58:03 ALPL202 kernel: [] __jbd2_log_wait_for_space+0xcf/0x1ad [jbd2] May 17 11:58:03 ALPL202 kernel: [] start_this_handle+0x346/0x3cf [jbd2] May 17 11:58:03 ALPL202 kernel: [] ldlm_lock_remove_from_lru+0x74/0xe0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] jbd2_journal_start+0xad/0xe1 [jbd2] May 17 11:58:03 ALPL202 kernel: [] fsfilt_ldiskfs_start+0x4c2/0x590 [fsfilt_ldiskfs] May 17 11:58:03 ALPL202 kernel: [] filter_version_get_check+0x91/0x2a0 [obdfilter] May 17 11:58:03 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:58:03 ALPL202 kernel: [] cfs_alloc+0x68/0xc0 [libcfs] May 17 11:58:03 ALPL202 kernel: [] filter_destroy+0xa2d/0x1f90 [obdfilter] May 17 11:58:03 ALPL202 kernel: [] ldlm_blocking_ast+0x0/0x2a0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ldlm_completion_ast+0x0/0x880 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] lustre_msg_add_version+0x34/0x110 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] lustre_pack_reply_flags+0x86a/0x950 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ldlm_resource_putref_internal+0x3ab/0x460 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ldlm_lock_put+0x372/0x3d0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] lustre_pack_reply+0x29/0xb0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ost_destroy+0x660/0x790 [ost] May 17 11:58:03 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] lustre_msg_get_opc+0x35/0xf0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ost_handle+0x1566/0x55c0 [ost] May 17 11:58:03 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 11:58:03 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:58:03 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:58:03 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: Pid: 6602, comm: ll_ost_io_63 May 17 11:58:03 ALPL202 kernel: May 17 11:58:03 ALPL202 kernel: Call Trace: May 17 11:58:03 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:58:03 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:58:03 ALPL202 kernel: [] __jbd2_log_wait_for_space+0x5d/0x1ad [jbd2] May 17 11:58:03 ALPL202 kernel: [] start_this_handle+0x346/0x3cf [jbd2] May 17 11:58:03 ALPL202 kernel: [] ldlm_lock_remove_from_lru+0x74/0xe0 [ptlrpc] May 17 11:58:03 ALPL202 kernel: [] jbd2_journal_start+0xad/0xe1 [jbd2] May 17 11:58:03 ALPL202 kernel: [] fsfilt_ldiskfs_start+0x4c2/0x590 [fsfilt_ldiskfs] May 17 11:58:03 ALPL202 kernel: [] filter_version_get_check+0x91/0x2a0 [obdfilter] May 17 11:58:03 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:58:03 ALPL202 kernel: [] cfs_alloc+0x68/0xc0 [libcfs] May 17 11:58:03 ALPL202 kernel: [] filter_destroy+0xa2d/0x1f90 [obdfilter] May 17 11:58:04 ALPL202 kernel: [] ldlm_blocking_ast+0x0/0x2a0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_completion_ast+0x0/0x880 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_add_version+0x34/0x110 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply_flags+0x86a/0x950 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_resource_putref_internal+0x3ab/0x460 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_lock_put+0x372/0x3d0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply+0x29/0xb0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_destroy+0x660/0x790 [ost] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_get_opc+0x35/0xf0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_handle+0x1566/0x55c0 [ost] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:58:04 ALPL202 kernel: May 17 11:58:04 ALPL202 kernel: Pid: 17476, comm: ll_ost_io_474 May 17 11:58:04 ALPL202 kernel: May 17 11:58:04 ALPL202 kernel: Call Trace: May 17 11:58:04 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:58:04 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:58:04 ALPL202 kernel: [] __jbd2_log_wait_for_space+0x5d/0x1ad [jbd2] May 17 11:58:04 ALPL202 kernel: [] start_this_handle+0x346/0x3cf [jbd2] May 17 11:58:04 ALPL202 kernel: [] ldlm_lock_remove_from_lru+0x74/0xe0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] jbd2_journal_start+0xad/0xe1 [jbd2] May 17 11:58:04 ALPL202 kernel: [] fsfilt_ldiskfs_start+0x4c2/0x590 [fsfilt_ldiskfs] May 17 11:58:04 ALPL202 kernel: [] filter_version_get_check+0x91/0x2a0 [obdfilter] May 17 11:58:04 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:58:04 ALPL202 kernel: [] cfs_alloc+0x68/0xc0 [libcfs] May 17 11:58:04 ALPL202 kernel: [] filter_destroy+0xa2d/0x1f90 [obdfilter] May 17 11:58:04 ALPL202 kernel: [] ldlm_blocking_ast+0x0/0x2a0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_completion_ast+0x0/0x880 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_add_version+0x34/0x110 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply_flags+0x86a/0x950 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_resource_putref_internal+0x3ab/0x460 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_lock_put+0x372/0x3d0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply+0x29/0xb0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_destroy+0x660/0x790 [ost] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_get_opc+0x35/0xf0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_handle+0x1566/0x55c0 [ost] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:58:04 ALPL202 kernel: May 17 11:58:04 ALPL202 kernel: Pid: 16683, comm: ll_ost_io_245 May 17 11:58:04 ALPL202 kernel: May 17 11:58:04 ALPL202 kernel: Call Trace: May 17 11:58:04 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 11:58:04 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 11:58:04 ALPL202 kernel: [] __jbd2_log_wait_for_space+0x5d/0x1ad [jbd2] May 17 11:58:04 ALPL202 kernel: [] start_this_handle+0x346/0x3cf [jbd2] May 17 11:58:04 ALPL202 kernel: [] ldlm_lock_remove_from_lru+0x74/0xe0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] jbd2_journal_start+0xad/0xe1 [jbd2] May 17 11:58:04 ALPL202 kernel: [] fsfilt_ldiskfs_start+0x4c2/0x590 [fsfilt_ldiskfs] May 17 11:58:04 ALPL202 kernel: [] filter_version_get_check+0x91/0x2a0 [obdfilter] May 17 11:58:04 ALPL202 kernel: [] __down_write_nested+0x12/0x92 May 17 11:58:04 ALPL202 kernel: [] cfs_alloc+0x68/0xc0 [libcfs] May 17 11:58:04 ALPL202 kernel: [] filter_destroy+0xa2d/0x1f90 [obdfilter] May 17 11:58:04 ALPL202 kernel: [] ldlm_blocking_ast+0x0/0x2a0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_completion_ast+0x0/0x880 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_add_version+0x34/0x110 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply_flags+0x86a/0x950 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_resource_putref_internal+0x3ab/0x460 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ldlm_lock_put+0x372/0x3d0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_pack_reply+0x29/0xb0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_destroy+0x660/0x790 [ost] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] lustre_msg_get_opc+0x35/0xf0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ost_handle+0x1566/0x55c0 [ost] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 11:58:04 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 11:58:04 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 11:58:04 ALPL202 kernel: May 17 11:58:04 ALPL202 kernel: Lustre: Service thread pid 17500 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:04 ALPL202 kernel: Lustre: Service thread pid 17320 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.16794 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.17329 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.6639 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.17320 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.17500 May 17 11:58:04 ALPL202 kernel: Lustre: Service thread pid 17484 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:04 ALPL202 kernel: Lustre: Skipped 3 previous similar messages May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.17484 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.16683 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.17476 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.6602 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.22745 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.6577 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835083.16721 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835084.16655 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835084.6647 May 17 11:58:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835084.6631 May 17 11:58:06 ALPL202 kernel: Lustre: Service thread pid 22746 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:06 ALPL202 kernel: Lustre: Skipped 5 previous similar messages May 17 11:58:06 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835086.22746 May 17 11:58:07 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835087.17469 May 17 11:58:07 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835087.17338 May 17 11:58:08 ALPL202 kernel: Lustre: Service thread pid 6626 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:08 ALPL202 kernel: Lustre: Skipped 2 previous similar messages May 17 11:58:08 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835088.6626 May 17 11:58:09 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835089.16722 May 17 11:58:09 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835089.6592 May 17 11:58:09 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835089.17319 May 17 11:58:10 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835090.17464 May 17 11:58:13 ALPL202 kernel: Lustre: Service thread pid 6550 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:13 ALPL202 kernel: Lustre: Skipped 4 previous similar messages May 17 11:58:13 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835093.6550 May 17 11:58:13 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835093.17472 May 17 11:58:16 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835096.17462 May 17 11:58:19 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835099.16803 May 17 11:58:19 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835099.17498 May 17 11:58:21 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835101.24011 May 17 11:58:24 ALPL202 kernel: Lustre: Service thread pid 17347 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:24 ALPL202 kernel: Lustre: Skipped 5 previous similar messages May 17 11:58:24 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835104.17347 May 17 11:58:24 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835104.17325 May 17 11:58:28 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835108.16652 May 17 11:58:28 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835108.6560 May 17 11:58:29 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835109.6657 May 17 11:58:30 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835110.16822 May 17 11:58:33 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835113.17341 May 17 11:58:33 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835113.16738 May 17 11:58:33 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835113.16659 May 17 11:58:38 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835118.6619 May 17 11:58:38 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835118.16795 May 17 11:58:38 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835118.6644 May 17 11:58:39 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835119.16723 May 17 11:58:42 ALPL202 kernel: Lustre: Service thread pid 7608 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:58:42 ALPL202 kernel: Lustre: Skipped 12 previous similar messages May 17 11:58:42 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835122.8132 May 17 11:58:42 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835122.7608 May 17 11:58:43 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835123.7536 May 17 11:58:43 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835123.7449 May 17 11:58:44 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835124.7358 May 17 11:58:45 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835125.7389 May 17 11:58:46 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835126.6512 May 17 11:58:47 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835127.7403 May 17 11:58:48 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835128.7577 May 17 11:58:49 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835129.7332 May 17 11:58:49 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835129.7291 May 17 11:58:50 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835130.7588 May 17 11:58:51 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835131.7533 May 17 11:58:52 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835132.7424 May 17 11:58:53 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835133.6440 May 17 11:58:54 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835134.7567 May 17 11:58:55 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835135.7495 May 17 11:58:55 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835135.6652 May 17 11:58:56 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835136.7289 May 17 11:58:57 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835137.6500 May 17 11:58:58 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835138.7284 May 17 11:58:59 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835139.6487 May 17 11:59:00 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835140.7573 May 17 11:59:01 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835141.7619 May 17 11:59:02 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835142.7442 May 17 11:59:03 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835143.7342 May 17 11:59:04 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835144.7500 May 17 11:59:05 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835145.6415 May 17 11:59:06 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835146.7569 May 17 11:59:06 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835146.17474 May 17 11:59:07 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835147.7566 May 17 11:59:08 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835148.7538 May 17 11:59:09 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835149.7601 May 17 11:59:10 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835150.6395 May 17 11:59:11 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835151.7527 May 17 11:59:12 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835152.7353 May 17 11:59:12 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835152.7343 May 17 11:59:13 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835153.7640 May 17 11:59:14 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835154.7305 May 17 11:59:15 ALPL202 kernel: Lustre: Service thread pid 6399 was inactive for 200.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 11:59:15 ALPL202 kernel: Lustre: Skipped 38 previous similar messages May 17 11:59:15 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835155.6399 May 17 11:59:16 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835156.6518 May 17 11:59:17 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835157.7479 May 17 11:59:18 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835158.6519 May 17 11:59:20 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835160.7355 May 17 11:59:21 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835161.7402 May 17 11:59:22 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835162.7579 May 17 11:59:23 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835163.7400 May 17 11:59:24 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835164.7613 May 17 11:59:25 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835165.7525 May 17 11:59:26 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835166.7650 May 17 11:59:27 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835167.6499 May 17 11:59:28 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835168.7583 May 17 11:59:29 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835169.7517 May 17 11:59:29 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835169.7390 May 17 11:59:30 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835170.7299 May 17 11:59:31 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835171.6532 May 17 11:59:32 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835172.7465 May 17 11:59:33 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835173.7335 May 17 11:59:34 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835174.7594 May 17 11:59:35 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835175.6436 May 17 11:59:36 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835176.6497 May 17 11:59:37 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835177.7320 May 17 11:59:38 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835178.7383 May 17 11:59:39 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835179.7489 May 17 11:59:40 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835180.7622 May 17 11:59:41 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835181.7518 May 17 11:59:42 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835182.6462 May 17 11:59:43 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835183.6427 May 17 12:01:09 ALPL202 kernel: Lustre: Service thread pid 17461 was inactive for 262.00s. Watchdog stack traces are limited to 3 per 300 seconds, skipping this one. May 17 12:01:09 ALPL202 kernel: Lustre: Skipped 28 previous similar messages May 17 12:01:09 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835269.17461 May 17 12:01:58 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431835318.17466 May 17 12:08:49 ALPL202 kernel: Lustre: 8912:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-207), not sending early reply May 17 12:08:49 ALPL202 kernel: req@ffff81062210b000 x1499839927858551/t0 o5->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 400/400 e 5 to 0 dl 1431835734 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:10:45 ALPL202 kernel: Lustre: 7347:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:10:45 ALPL202 kernel: Lustre: 7347:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 1 previous similar message May 17 12:10:45 ALPL202 kernel: Lustre: 7347:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:10:45 ALPL202 kernel: LustreError: 7347:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103c80b9400 x1499839927900702/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431835945 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:10:45 ALPL202 kernel: LustreError: 7347:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 1159 previous similar messages May 17 12:10:52 ALPL202 kernel: Lustre: 7368:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:10:52 ALPL202 kernel: LustreError: 7368:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810622525c00 x1499839927900703/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431835952 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:10:59 ALPL202 kernel: Lustre: 7406:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:10:59 ALPL202 kernel: Lustre: 7406:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 1 previous similar message May 17 12:10:59 ALPL202 kernel: Lustre: 7406:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:11:06 ALPL202 kernel: Lustre: 6564:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:06 ALPL202 kernel: req@ffff8105e1918c00 x1499653315101158/t0 o4->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 448/416 e 4 to 0 dl 1431835871 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:06 ALPL202 kernel: LustreError: 6423:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102ccdfac00 x1499839927900725/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431835966 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:11:06 ALPL202 kernel: LustreError: 6423:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 1 previous similar message May 17 12:11:07 ALPL202 kernel: Lustre: 16412:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:07 ALPL202 kernel: req@ffff8105b1e82400 x1497754013855499/t0 o4->4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@NET_0x500000a030111_UUID:0/0 lens 448/416 e 4 to 0 dl 1431835872 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:07 ALPL202 kernel: Lustre: 16412:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 12 previous similar messages May 17 12:11:09 ALPL202 kernel: Lustre: 6559:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:09 ALPL202 kernel: req@ffff8104dfe75c00 x1486456502180484/t0 o4->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 448/416 e 4 to 0 dl 1431835874 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:09 ALPL202 kernel: Lustre: 6559:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 2 previous similar messages May 17 12:11:12 ALPL202 kernel: Lustre: 16764:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:12 ALPL202 kernel: req@ffff8103907fd000 x1498234008126971/t0 o4->ea79a342-578d-a338-caef-c51d0fe5e904@NET_0x500000a03012e_UUID:0/0 lens 448/416 e 4 to 0 dl 1431835877 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:12 ALPL202 kernel: Lustre: 16764:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 3 previous similar messages May 17 12:11:13 ALPL202 kernel: Lustre: 7590:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:11:13 ALPL202 kernel: Lustre: 7590:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 1 previous similar message May 17 12:11:19 ALPL202 kernel: Lustre: 6576:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:19 ALPL202 kernel: req@ffff8105928ab000 x1499006743174487/t0 o4->ada59592-d805-c261-0c63-be0a8c2fa922@NET_0x500000a03011f_UUID:0/0 lens 448/416 e 4 to 0 dl 1431835884 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:19 ALPL202 kernel: Lustre: 6576:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 5 previous similar messages May 17 12:11:20 ALPL202 kernel: Lustre: 7552:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:11:20 ALPL202 kernel: Lustre: 7552:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2 previous similar messages May 17 12:11:27 ALPL202 kernel: LustreError: 6470:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e11bc000 x1499839927900834/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431835987 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:11:27 ALPL202 kernel: LustreError: 6470:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2 previous similar messages May 17 12:11:31 ALPL202 kernel: Lustre: 6611:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:31 ALPL202 kernel: req@ffff8102f6f0dc00 x1486619728712225/t0 o4->ab1586ab-f630-d76b-7485-396b47d527b1@NET_0x500000a030107_UUID:0/0 lens 448/416 e 4 to 0 dl 1431835896 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:31 ALPL202 kernel: Lustre: 6611:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 5 previous similar messages May 17 12:11:34 ALPL202 kernel: Lustre: 6412:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:11:34 ALPL202 kernel: Lustre: 6412:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2 previous similar messages May 17 12:11:47 ALPL202 kernel: Lustre: 7474:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-383), not sending early reply May 17 12:11:47 ALPL202 kernel: req@ffff8104e44e8c00 x1497853921904206/t0 o101->efebfe6f-6429-bcd2-fd37-e715eaf8983b@NET_0x500000a030124_UUID:0/0 lens 296/0 e 4 to 0 dl 1431835912 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:11:47 ALPL202 kernel: Lustre: 7474:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 13 previous similar messages May 17 12:11:55 ALPL202 kernel: Lustre: 7633:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:11:55 ALPL202 kernel: Lustre: 7633:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 4 previous similar messages May 17 12:12:02 ALPL202 kernel: LustreError: 7494:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106093adc00 x1499839927900873/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431836022 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:12:02 ALPL202 kernel: LustreError: 7494:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 4 previous similar messages May 17 12:12:09 ALPL202 kernel: Lustre: 7283:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:12:09 ALPL202 kernel: Lustre: 7283:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 4 previous similar messages May 17 12:12:20 ALPL202 kernel: Lustre: 7528:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-367), not sending early reply May 17 12:12:20 ALPL202 kernel: req@ffff81038f446000 x1485800369979002/t0 o101->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 296/0 e 3 to 0 dl 1431835945 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:12:20 ALPL202 kernel: Lustre: 7528:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 48 previous similar messages May 17 12:13:05 ALPL202 kernel: Lustre: 6478:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:13:05 ALPL202 kernel: Lustre: 6478:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 9 previous similar messages May 17 12:13:12 ALPL202 kernel: LustreError: 7546:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104c627bc00 x1499839927901043/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431836092 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:13:12 ALPL202 kernel: LustreError: 7546:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 9 previous similar messages May 17 12:13:19 ALPL202 kernel: Lustre: 7530:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 12:13:19 ALPL202 kernel: Lustre: 7530:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 9 previous similar messages May 17 12:15:14 ALPL202 kernel: Lustre: 7475:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 17 12:15:14 ALPL202 kernel: Lustre: 7475:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 421 previous similar messages May 17 12:15:21 ALPL202 kernel: LustreError: 7376:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105bd560050 x1497851289700314/t0 o8->d7a5a345-9841-f534-ddde-a2808cdd637c@:0/0 lens 368/264 e 0 to 0 dl 1431836221 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:15:21 ALPL202 kernel: LustreError: 7376:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 446 previous similar messages May 17 12:15:27 ALPL202 kernel: Lustre: 7381:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ea79a342-578d-a338-caef-c51d0fe5e904@10.3.1.46@o2ib to 0xffff810372190200; still busy with 2 active RPCs May 17 12:15:27 ALPL202 kernel: Lustre: 7381:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 474 previous similar messages May 17 12:19:30 ALPL202 kernel: Lustre: 7658:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 12:19:30 ALPL202 kernel: Lustre: 7658:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 1148 previous similar messages May 17 12:19:37 ALPL202 kernel: LustreError: 7435:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81007d79b000 x1486456502232612/t0 o8->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 368/264 e 0 to 0 dl 1431836477 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:19:37 ALPL202 kernel: LustreError: 7435:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 1157 previous similar messages May 17 12:19:44 ALPL202 kernel: Lustre: 6457:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 127a567c-2e83-70ca-5c5a-99111e1a5592@10.3.1.14@o2ib to 0xffff8105e5c98600; still busy with 5 active RPCs May 17 12:19:44 ALPL202 kernel: Lustre: 6457:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 1157 previous similar messages May 17 12:22:29 ALPL202 kernel: Lustre: 20857:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-36), not sending early reply May 17 12:22:29 ALPL202 kernel: req@ffff81028b284c00 x1487072294624022/t0 o4->86d0ab2a-246f-a784-2744-c50f68cee379@NET_0x500000a03011a_UUID:0/0 lens 448/416 e 0 to 0 dl 1431836554 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:22:29 ALPL202 kernel: Lustre: 20857:0:(service.c:812:ptlrpc_at_send_early_reply()) Skipped 16 previous similar messages May 17 12:26:46 ALPL202 kernel: Lustre: Service thread pid 6665 was inactive for 1018.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: May 17 12:26:46 ALPL202 kernel: Lustre: Skipped 3 previous similar messages May 17 12:26:46 ALPL202 kernel: Pid: 6665, comm: ll_ost_io_126 May 17 12:26:46 ALPL202 kernel: May 17 12:26:46 ALPL202 kernel: Call Trace: May 17 12:26:46 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 12:26:46 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 12:26:46 ALPL202 kernel: [] __jbd2_log_wait_for_space+0x5d/0x1ad [jbd2] May 17 12:26:46 ALPL202 kernel: [] start_this_handle+0x346/0x3cf [jbd2] May 17 12:26:46 ALPL202 kernel: [] cache_alloc_refill+0x108/0x188 May 17 12:26:46 ALPL202 kernel: [] jbd2_journal_start+0xad/0xe1 [jbd2] May 17 12:26:46 ALPL202 kernel: [] fsfilt_ldiskfs_brw_start+0x35c/0x490 [fsfilt_ldiskfs] May 17 12:26:46 ALPL202 kernel: [] filter_quota_acquire+0x0/0x120 [lquota] May 17 12:26:46 ALPL202 kernel: [] filter_commitrw_write+0xe4d/0x2e70 [obdfilter] May 17 12:26:46 ALPL202 kernel: [] ost_checksum_bulk+0x2c9/0x5a0 [ost] May 17 12:26:46 ALPL202 kernel: [] ost_brw_write+0x1c99/0x2480 [ost] May 17 12:26:46 ALPL202 kernel: [] ptlrpc_send_reply+0x5e8/0x600 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] target_committed_to_req+0x40/0x120 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 12:26:46 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] ost_handle+0x2bc2/0x55c0 [ost] May 17 12:26:46 ALPL202 kernel: [] class_handle2object+0xe0/0x170 [obdclass] May 17 12:26:46 ALPL202 kernel: [] lock_res_and_lock+0xba/0xd0 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] __ldlm_handle2lock+0x2f8/0x360 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 12:26:46 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 12:26:46 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 12:26:46 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 12:26:46 ALPL202 kernel: May 17 12:26:46 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431836806.6665 May 17 12:28:03 ALPL202 kernel: Lustre: 6419:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 12:28:03 ALPL202 kernel: Lustre: 6419:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2311 previous similar messages May 17 12:28:08 ALPL202 kernel: Lustre: 6545:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-500), not sending early reply May 17 12:28:08 ALPL202 kernel: req@ffff8102a19f4050 x1487068466817766/t0 o4->2a3fd8ac-ad9e-6c5c-eb5a-e9e64491e0ac@NET_0x500000a030128_UUID:0/0 lens 448/416 e 2 to 0 dl 1431836893 ref 2 fl Interpret:/0/0 rc 0/0 May 17 12:28:09 ALPL202 kernel: LustreError: 7605:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104bec49400 x1487072294639195/t0 o8->86d0ab2a-246f-a784-2744-c50f68cee379@NET_0x500000a03011a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431836989 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:28:09 ALPL202 kernel: LustreError: 7605:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2305 previous similar messages May 17 12:28:16 ALPL202 kernel: Lustre: 7460:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from b723915e-bd6d-5de8-3f38-d333eb1db2ed@10.3.1.3@o2ib to 0xffff81036846c200; still busy with 2 active RPCs May 17 12:28:16 ALPL202 kernel: Lustre: 7460:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2305 previous similar messages May 17 12:31:53 ALPL202 kernel: Lustre: Service thread pid 16697 was inactive for 1200.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: May 17 12:31:53 ALPL202 kernel: Pid: 16697, comm: ll_ost_io_259 May 17 12:31:53 ALPL202 kernel: May 17 12:31:53 ALPL202 kernel: Call Trace: May 17 12:31:53 ALPL202 kernel: [] ldiskfs_get_blocks+0xaa/0x210 [ldiskfs] May 17 12:31:53 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 12:31:53 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 12:31:53 ALPL202 kernel: [] filter_commitrw_write+0xc47/0x2e70 [obdfilter] May 17 12:31:53 ALPL202 kernel: [] ost_checksum_bulk+0x360/0x5a0 [ost] May 17 12:31:53 ALPL202 kernel: [] ost_brw_write+0x1c99/0x2480 [ost] May 17 12:31:53 ALPL202 kernel: [] ptlrpc_send_reply+0x5e8/0x600 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] target_committed_to_req+0x40/0x120 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] default_wake_function+0x0/0xe May 17 12:31:53 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] ost_handle+0x2bc2/0x55c0 [ost] May 17 12:31:53 ALPL202 kernel: [] class_handle2object+0xe0/0x170 [obdclass] May 17 12:31:53 ALPL202 kernel: [] lock_res_and_lock+0xba/0xd0 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] __ldlm_handle2lock+0x2f8/0x360 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 12:31:53 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 12:31:53 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 12:31:53 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 12:31:53 ALPL202 kernel: May 17 12:31:53 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431837113.16697 May 17 12:38:03 ALPL202 kernel: Lustre: 7382:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f9fdf5c1-f5d4-d701-81d8-f6afd06169ef reconnecting May 17 12:38:03 ALPL202 kernel: Lustre: 7382:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2753 previous similar messages May 17 12:38:09 ALPL202 kernel: LustreError: 7280:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102d772e850 x1499689912023786/t0 o8->ade1e689-13ba-5165-30df-204eabc4dae8@NET_0x500000a030112_UUID:0/0 lens 368/264 e 0 to 0 dl 1431837589 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:38:09 ALPL202 kernel: LustreError: 7280:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2750 previous similar messages May 17 12:38:16 ALPL202 kernel: Lustre: 7657:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ea79a342-578d-a338-caef-c51d0fe5e904@10.3.1.46@o2ib to 0xffff810372190200; still busy with 2 active RPCs May 17 12:38:16 ALPL202 kernel: Lustre: 7657:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2752 previous similar messages May 17 12:44:47 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 12:48:03 ALPL202 kernel: Lustre: 6433:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e8917316-444f-9483-db5a-7c7fb1549a1e reconnecting May 17 12:48:03 ALPL202 kernel: Lustre: 6433:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2768 previous similar messages May 17 12:48:11 ALPL202 kernel: LustreError: 7564:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810577277c00 x1499596989803193/t0 o8->f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@NET_0x500000a030116_UUID:0/0 lens 368/264 e 0 to 0 dl 1431838191 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:48:11 ALPL202 kernel: LustreError: 7564:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2772 previous similar messages May 17 12:48:16 ALPL202 kernel: Lustre: 7543:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@10.3.1.17@o2ib to 0xffff81061f5f1200; still busy with 4 active RPCs May 17 12:48:16 ALPL202 kernel: Lustre: 7543:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2772 previous similar messages May 17 12:58:03 ALPL202 kernel: Lustre: 6414:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d3e32527-f91f-d191-fbe6-fbc0e0ba6beb reconnecting May 17 12:58:03 ALPL202 kernel: Lustre: 6414:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2771 previous similar messages May 17 12:58:11 ALPL202 kernel: LustreError: 6456:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104a25c8800 x1485800792475611/t0 o8->dee84236-56bc-f2f7-cd2c-ba3d555fd617@NET_0x500000a03011b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431838791 ref 1 fl Interpret:/0/0 rc -16/0 May 17 12:58:11 ALPL202 kernel: LustreError: 6456:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2775 previous similar messages May 17 12:58:16 ALPL202 kernel: Lustre: 7303:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 17 12:58:16 ALPL202 kernel: Lustre: 7303:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2766 previous similar messages May 17 13:08:04 ALPL202 kernel: Lustre: 7318:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 17 13:08:04 ALPL202 kernel: Lustre: 7318:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2769 previous similar messages May 17 13:08:11 ALPL202 kernel: LustreError: 7341:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105de142800 x1498432867515189/t0 o8->de0fa4c5-b63e-b22c-6d6a-d2383f854657@NET_0x500000a030114_UUID:0/0 lens 368/264 e 0 to 0 dl 1431839391 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:08:11 ALPL202 kernel: LustreError: 7341:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2764 previous similar messages May 17 13:08:16 ALPL202 kernel: Lustre: 7352:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 13:08:16 ALPL202 kernel: Lustre: 7352:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2770 previous similar messages May 17 13:18:04 ALPL202 kernel: Lustre: 6450:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: efebfe6f-6429-bcd2-fd37-e715eaf8983b reconnecting May 17 13:18:04 ALPL202 kernel: Lustre: 6450:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2767 previous similar messages May 17 13:18:11 ALPL202 kernel: LustreError: 7308:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81044963c050 x1499839927909238/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431839991 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:18:11 ALPL202 kernel: LustreError: 7308:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2773 previous similar messages May 17 13:18:17 ALPL202 kernel: Lustre: 7434:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from dee84236-56bc-f2f7-cd2c-ba3d555fd617@10.3.1.27@o2ib to 0xffff81050f727a00; still busy with 2 active RPCs May 17 13:18:17 ALPL202 kernel: Lustre: 7434:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2772 previous similar messages May 17 13:28:04 ALPL202 kernel: Lustre: 7387:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e986becc-307e-f48d-480a-444f9c6b233c reconnecting May 17 13:28:04 ALPL202 kernel: Lustre: 7387:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 13:28:12 ALPL202 kernel: LustreError: 6412:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81022939b400 x1499682232320457/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431840592 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:28:12 ALPL202 kernel: LustreError: 6412:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2770 previous similar messages May 17 13:28:17 ALPL202 kernel: Lustre: 7602:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab1586ab-f630-d76b-7485-396b47d527b1@10.3.1.7@o2ib to 0xffff8105db9f0600; still busy with 3 active RPCs May 17 13:28:17 ALPL202 kernel: Lustre: 7602:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2765 previous similar messages May 17 13:38:04 ALPL202 kernel: Lustre: 7377:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4996fcc3-85b7-659a-e4a1-572255254f65 reconnecting May 17 13:38:04 ALPL202 kernel: Lustre: 7377:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2769 previous similar messages May 17 13:38:12 ALPL202 kernel: LustreError: 6526:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8100b4335c00 x1499653315354636/t0 o8->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 368/264 e 0 to 0 dl 1431841192 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:38:12 ALPL202 kernel: LustreError: 6526:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2768 previous similar messages May 17 13:38:17 ALPL202 kernel: Lustre: 6457:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@10.3.1.17@o2ib to 0xffff81061f5f1200; still busy with 4 active RPCs May 17 13:38:17 ALPL202 kernel: Lustre: 6457:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2773 previous similar messages May 17 13:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 13:48:04 ALPL202 kernel: Lustre: 7308:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e8917316-444f-9483-db5a-7c7fb1549a1e reconnecting May 17 13:48:04 ALPL202 kernel: Lustre: 7308:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2766 previous similar messages May 17 13:48:12 ALPL202 kernel: LustreError: 6513:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106224b6400 x1499017963672276/t0 o8->1ed5a549-efbb-515c-ff9a-8b8741af8256@NET_0x500000a030127_UUID:0/0 lens 368/264 e 0 to 0 dl 1431841792 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:48:12 ALPL202 kernel: LustreError: 6513:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2772 previous similar messages May 17 13:48:17 ALPL202 kernel: Lustre: 7458:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 13:48:17 ALPL202 kernel: Lustre: 7458:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2769 previous similar messages May 17 13:58:05 ALPL202 kernel: Lustre: 7586:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0 reconnecting May 17 13:58:05 ALPL202 kernel: Lustre: 7586:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 13:58:12 ALPL202 kernel: LustreError: 7549:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105ab242400 x1499018006799253/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431842392 ref 1 fl Interpret:/0/0 rc -16/0 May 17 13:58:12 ALPL202 kernel: LustreError: 7549:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2765 previous similar messages May 17 13:58:18 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ea79a342-578d-a338-caef-c51d0fe5e904@10.3.1.46@o2ib to 0xffff810372190200; still busy with 2 active RPCs May 17 13:58:18 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2768 previous similar messages May 17 14:08:05 ALPL202 kernel: Lustre: 6397:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 17 14:08:05 ALPL202 kernel: Lustre: 6397:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 14:08:12 ALPL202 kernel: LustreError: 7418:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101d4f8e850 x1498234008382585/t0 o8->ea79a342-578d-a338-caef-c51d0fe5e904@NET_0x500000a03012e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431842992 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:08:12 ALPL202 kernel: LustreError: 7418:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2769 previous similar messages May 17 14:08:18 ALPL202 kernel: Lustre: 7352:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e986becc-307e-f48d-480a-444f9c6b233c@10.3.1.35@o2ib to 0xffff81017d1a0600; still busy with 4 active RPCs May 17 14:08:18 ALPL202 kernel: Lustre: 7352:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2772 previous similar messages May 17 14:18:05 ALPL202 kernel: Lustre: 7376:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84 reconnecting May 17 14:18:05 ALPL202 kernel: Lustre: 7376:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2766 previous similar messages May 17 14:18:12 ALPL202 kernel: LustreError: 7405:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106317b6400 x1497754014198412/t0 o8->4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@NET_0x500000a030111_UUID:0/0 lens 368/264 e 0 to 0 dl 1431843592 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:18:12 ALPL202 kernel: LustreError: 7405:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2769 previous similar messages May 17 14:18:18 ALPL202 kernel: Lustre: 7514:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 17 14:18:18 ALPL202 kernel: Lustre: 7514:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2770 previous similar messages May 17 14:28:06 ALPL202 kernel: Lustre: 7302:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d7a5a345-9841-f534-ddde-a2808cdd637c reconnecting May 17 14:28:06 ALPL202 kernel: Lustre: 7302:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 14:28:12 ALPL202 kernel: LustreError: 6422:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81004d5ce000 x1499010341352518/t0 o8->02af742f-5725-28cc-1be3-577b0076f6d3@NET_0x500000a03012c_UUID:0/0 lens 368/264 e 0 to 0 dl 1431844192 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:28:12 ALPL202 kernel: LustreError: 6422:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2773 previous similar messages May 17 14:28:18 ALPL202 kernel: Lustre: 7580:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ade1e689-13ba-5165-30df-204eabc4dae8@10.3.1.18@o2ib to 0xffff81055247fa00; still busy with 3 active RPCs May 17 14:28:18 ALPL202 kernel: Lustre: 7580:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2766 previous similar messages May 17 14:38:06 ALPL202 kernel: Lustre: 6418:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 17 14:38:06 ALPL202 kernel: Lustre: 6418:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2770 previous similar messages May 17 14:38:13 ALPL202 kernel: LustreError: 7647:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810621d0c400 x1499033020680420/t0 o8->79a59854-31b9-d753-4e22-8912905a5085@:0/0 lens 368/264 e 0 to 0 dl 1431844793 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:38:13 ALPL202 kernel: LustreError: 7647:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2768 previous similar messages May 17 14:38:18 ALPL202 kernel: Lustre: 7448:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17e28ade-58ed-211a-39c9-291989587703@10.3.1.29@o2ib to 0xffff81016379be00; still busy with 2 active RPCs May 17 14:38:18 ALPL202 kernel: Lustre: 7448:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2771 previous similar messages May 17 14:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 14:48:06 ALPL202 kernel: Lustre: 7340:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 86d0ab2a-246f-a784-2744-c50f68cee379 reconnecting May 17 14:48:06 ALPL202 kernel: Lustre: 7340:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2767 previous similar messages May 17 14:48:13 ALPL202 kernel: LustreError: 7425:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103bb342c00 x1485802397791299/t0 o8->72657e2e-01a5-e000-c945-6edb35b026e6@NET_0x500000a03012a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431845393 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:48:13 ALPL202 kernel: LustreError: 7425:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2771 previous similar messages May 17 14:48:18 ALPL202 kernel: Lustre: 7543:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 02af742f-5725-28cc-1be3-577b0076f6d3@10.3.1.44@o2ib to 0xffff81050f727e00; still busy with 1 active RPCs May 17 14:48:18 ALPL202 kernel: Lustre: 7543:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2769 previous similar messages May 17 14:58:06 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 2a3fd8ac-ad9e-6c5c-eb5a-e9e64491e0ac reconnecting May 17 14:58:06 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 14:58:13 ALPL202 kernel: LustreError: 7463:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810353664000 x1498183789992116/t0 o8->4996fcc3-85b7-659a-e4a1-572255254f65@NET_0x500000a030119_UUID:0/0 lens 368/264 e 0 to 0 dl 1431845993 ref 1 fl Interpret:/0/0 rc -16/0 May 17 14:58:13 ALPL202 kernel: LustreError: 7463:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2771 previous similar messages May 17 14:58:18 ALPL202 kernel: Lustre: 6486:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d3e32527-f91f-d191-fbe6-fbc0e0ba6beb@10.3.1.13@o2ib to 0xffff81021edfda00; still busy with 1 active RPCs May 17 14:58:18 ALPL202 kernel: Lustre: 6486:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2768 previous similar messages May 17 15:08:06 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84 reconnecting May 17 15:08:06 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2765 previous similar messages May 17 15:08:13 ALPL202 kernel: LustreError: 6476:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103354d8800 x1485802397823512/t0 o8->72657e2e-01a5-e000-c945-6edb35b026e6@NET_0x500000a03012a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431846593 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:08:13 ALPL202 kernel: LustreError: 6476:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2764 previous similar messages May 17 15:08:19 ALPL202 kernel: Lustre: 6495:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04cfad47-91cb-7688-6409-2530587af605@10.3.1.48@o2ib to 0xffff810479a1a600; still busy with 2 active RPCs May 17 15:08:19 ALPL202 kernel: Lustre: 6495:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2770 previous similar messages May 17 15:18:06 ALPL202 kernel: Lustre: 6491:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ada59592-d805-c261-0c63-be0a8c2fa922 reconnecting May 17 15:18:06 ALPL202 kernel: Lustre: 6491:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2772 previous similar messages May 17 15:18:13 ALPL202 kernel: LustreError: 7476:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103af478400 x1497851379833679/t0 o8->e986becc-307e-f48d-480a-444f9c6b233c@NET_0x500000a030123_UUID:0/0 lens 368/264 e 0 to 0 dl 1431847193 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:18:13 ALPL202 kernel: LustreError: 7476:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2771 previous similar messages May 17 15:18:19 ALPL202 kernel: Lustre: 7630:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from b723915e-bd6d-5de8-3f38-d333eb1db2ed@10.3.1.3@o2ib to 0xffff81036846c200; still busy with 2 active RPCs May 17 15:18:19 ALPL202 kernel: Lustre: 7630:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2766 previous similar messages May 17 15:28:06 ALPL202 kernel: Lustre: 7437:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 15:28:06 ALPL202 kernel: Lustre: 7437:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2771 previous similar messages May 17 15:28:13 ALPL202 kernel: LustreError: 7516:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8100b4335000 x1499775196633167/t0 o8->f20e7e2b-9669-0314-8db8-3d5538bfde76@:0/0 lens 368/264 e 0 to 0 dl 1431847793 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:28:13 ALPL202 kernel: LustreError: 7516:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2765 previous similar messages May 17 15:28:19 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@10.3.1.32@o2ib to 0xffff810c3cf28000; still busy with 3 active RPCs May 17 15:28:19 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2772 previous similar messages May 17 15:38:06 ALPL202 kernel: Lustre: 7581:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 929aa131-7e43-4b34-d056-6e36a0b13e1a reconnecting May 17 15:38:06 ALPL202 kernel: Lustre: 7581:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2762 previous similar messages May 17 15:38:14 ALPL202 kernel: LustreError: 7328:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810045c88000 x1485802397871702/t0 o8->72657e2e-01a5-e000-c945-6edb35b026e6@NET_0x500000a03012a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431848394 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:38:14 ALPL202 kernel: LustreError: 7328:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2774 previous similar messages May 17 15:38:19 ALPL202 kernel: Lustre: 6470:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f0bae738-4650-c254-ee38-ca124e1eea57@10.3.1.11@o2ib to 0xffff810517df7e00; still busy with 4 active RPCs May 17 15:38:19 ALPL202 kernel: Lustre: 6470:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2770 previous similar messages May 17 15:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 15:48:06 ALPL202 kernel: Lustre: 6424:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 1ed5a549-efbb-515c-ff9a-8b8741af8256 reconnecting May 17 15:48:06 ALPL202 kernel: Lustre: 6424:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2774 previous similar messages May 17 15:48:15 ALPL202 kernel: LustreError: 7366:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e3eb5800 x1499018007072157/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431848995 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:48:15 ALPL202 kernel: LustreError: 7366:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2775 previous similar messages May 17 15:48:19 ALPL202 kernel: Lustre: 7410:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from b723915e-bd6d-5de8-3f38-d333eb1db2ed@10.3.1.3@o2ib to 0xffff81036846c200; still busy with 2 active RPCs May 17 15:48:19 ALPL202 kernel: Lustre: 7410:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2761 previous similar messages May 17 15:58:07 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 17 15:58:07 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2771 previous similar messages May 17 15:58:15 ALPL202 kernel: LustreError: 7559:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81063ed6c000 x1499645795663173/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431849595 ref 1 fl Interpret:/0/0 rc -16/0 May 17 15:58:15 ALPL202 kernel: LustreError: 7559:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2765 previous similar messages May 17 15:58:19 ALPL202 kernel: Lustre: 7653:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 17 15:58:19 ALPL202 kernel: Lustre: 7653:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2775 previous similar messages May 17 16:08:07 ALPL202 kernel: Lustre: 7378:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 72657e2e-01a5-e000-c945-6edb35b026e6 reconnecting May 17 16:08:07 ALPL202 kernel: Lustre: 7378:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2761 previous similar messages May 17 16:08:15 ALPL202 kernel: LustreError: 7607:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101f7187800 x1499682232691385/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431850195 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:08:15 ALPL202 kernel: LustreError: 7607:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2770 previous similar messages May 17 16:08:19 ALPL202 kernel: Lustre: 7304:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 17 16:08:19 ALPL202 kernel: Lustre: 7304:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2773 previous similar messages May 17 16:18:07 ALPL202 kernel: Lustre: 6423:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 1ed5a549-efbb-515c-ff9a-8b8741af8256 reconnecting May 17 16:18:07 ALPL202 kernel: Lustre: 6423:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2777 previous similar messages May 17 16:18:15 ALPL202 kernel: LustreError: 7300:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105043c3400 x1499649574229228/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431850795 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:18:15 ALPL202 kernel: LustreError: 7300:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2773 previous similar messages May 17 16:18:19 ALPL202 kernel: Lustre: 7430:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 16:18:19 ALPL202 kernel: Lustre: 7430:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2762 previous similar messages May 17 16:28:08 ALPL202 kernel: Lustre: 7599:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f9ed569f-59c6-0f68-7b91-ef32fbaef2d8 reconnecting May 17 16:28:08 ALPL202 kernel: Lustre: 7599:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2766 previous similar messages May 17 16:28:15 ALPL202 kernel: LustreError: 7298:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81062252a400 x1499653315762300/t0 o8->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 368/264 e 0 to 0 dl 1431851395 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:28:15 ALPL202 kernel: LustreError: 7298:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2761 previous similar messages May 17 16:28:19 ALPL202 kernel: Lustre: 6504:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 1ed5a549-efbb-515c-ff9a-8b8741af8256@10.3.1.39@o2ib to 0xffff81050f727600; still busy with 1 active RPCs May 17 16:28:19 ALPL202 kernel: Lustre: 6504:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2774 previous similar messages May 17 16:38:08 ALPL202 kernel: Lustre: 6478:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d7a5a345-9841-f534-ddde-a2808cdd637c reconnecting May 17 16:38:08 ALPL202 kernel: Lustre: 6478:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2768 previous similar messages May 17 16:38:15 ALPL202 kernel: LustreError: 7503:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105dd8fa800 x1486456502737521/t0 o8->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 368/264 e 0 to 0 dl 1431851995 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:38:15 ALPL202 kernel: LustreError: 7503:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2772 previous similar messages May 17 16:38:20 ALPL202 kernel: Lustre: 7541:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@10.3.1.22@o2ib to 0xffff81063fe3a400; still busy with 1 active RPCs May 17 16:38:20 ALPL202 kernel: Lustre: 7541:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2765 previous similar messages May 17 16:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 16:48:08 ALPL202 kernel: Lustre: 6403:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 16:48:08 ALPL202 kernel: Lustre: 6403:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2778 previous similar messages May 17 16:48:15 ALPL202 kernel: LustreError: 7654:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103fe3bd800 x1497853922634667/t0 o8->efebfe6f-6429-bcd2-fd37-e715eaf8983b@NET_0x500000a030124_UUID:0/0 lens 368/264 e 0 to 0 dl 1431852595 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:48:15 ALPL202 kernel: LustreError: 7654:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2771 previous similar messages May 17 16:48:20 ALPL202 kernel: Lustre: 6437:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@10.3.1.17@o2ib to 0xffff81061f5f1200; still busy with 4 active RPCs May 17 16:48:20 ALPL202 kernel: Lustre: 6437:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2768 previous similar messages May 17 16:58:08 ALPL202 kernel: Lustre: 7483:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 17342f99-c92f-d6af-0247-fdddc176fe9b reconnecting May 17 16:58:08 ALPL202 kernel: Lustre: 7483:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2766 previous similar messages May 17 16:58:15 ALPL202 kernel: LustreError: 6412:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81044a41f000 x1499566480199795/t0 o8->f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@NET_0x500000a03012b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431853195 ref 1 fl Interpret:/0/0 rc -16/0 May 17 16:58:15 ALPL202 kernel: LustreError: 6412:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2764 previous similar messages May 17 16:58:20 ALPL202 kernel: Lustre: 6465:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f32cf79c-e087-bfe3-6e8f-7fde95ab168b@10.3.1.15@o2ib to 0xffff8105c33c4e00; still busy with 3 active RPCs May 17 16:58:20 ALPL202 kernel: Lustre: 6465:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2777 previous similar messages May 17 16:59:05 ALPL202 kernel: Lustre: Service thread pid 7524 was inactive for 200.00s. The thread might be hung, or it might only be slow and will resume later. Dumping the stack trace for debugging purposes: May 17 16:59:05 ALPL202 kernel: Pid: 7524, comm: ll_ost_374 May 17 16:59:05 ALPL202 kernel: May 17 16:59:05 ALPL202 kernel: Call Trace: May 17 16:59:05 ALPL202 kernel: [] __mutex_lock_slowpath+0x60/0x9b May 17 16:59:05 ALPL202 kernel: [] .text.lock.mutex+0xf/0x14 May 17 16:59:05 ALPL202 kernel: [] filter_parent_lock+0x36/0x220 [obdfilter] May 17 16:59:05 ALPL202 kernel: [] lnet_ni_send+0x96/0xe0 [lnet] May 17 16:59:05 ALPL202 kernel: [] filter_fid2dentry+0x2cf/0x740 [obdfilter] May 17 16:59:05 ALPL202 kernel: [] __down_trylock+0x44/0x4e May 17 16:59:05 ALPL202 kernel: [] filter_lvbo_init+0x3bb/0x69b [obdfilter] May 17 16:59:05 ALPL202 kernel: [] ldlm_resource_get+0x900/0xa60 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ost_blocking_ast+0x0/0x9b0 [ost] May 17 16:59:05 ALPL202 kernel: [] ldlm_server_completion_ast+0x0/0x5e0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ldlm_lock_create+0xba/0x9f0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] lustre_swab_buf+0x81/0x170 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ldlm_server_glimpse_ast+0x0/0x3b0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ldlm_server_glimpse_ast+0x0/0x3b0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ldlm_server_completion_ast+0x0/0x5e0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ost_blocking_ast+0x0/0x9b0 [ost] May 17 16:59:05 ALPL202 kernel: [] ldlm_handle_enqueue+0x670/0x1210 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] lustre_msg_check_version_v2+0x8/0x20 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ost_handle+0x5003/0x55c0 [ost] May 17 16:59:05 ALPL202 kernel: [] ptlrpc_server_handle_request+0x984/0xe00 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] ptlrpc_wait_event+0x2e5/0x310 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] __wake_up_common+0x3e/0x68 May 17 16:59:05 ALPL202 kernel: [] ptlrpc_main+0xf16/0x10e0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] child_rip+0xa/0x11 May 17 16:59:05 ALPL202 kernel: [] ptlrpc_main+0x0/0x10e0 [ptlrpc] May 17 16:59:05 ALPL202 kernel: [] child_rip+0x0/0x11 May 17 16:59:05 ALPL202 kernel: May 17 16:59:05 ALPL202 kernel: LustreError: dumping log to /tmp/lustre-log.1431853145.7524 May 17 17:08:08 ALPL202 kernel: Lustre: 7336:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 17:08:08 ALPL202 kernel: Lustre: 7336:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2766 previous similar messages May 17 17:08:15 ALPL202 kernel: LustreError: 7351:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102fe2a7800 x1499010341766547/t0 o8->02af742f-5725-28cc-1be3-577b0076f6d3@NET_0x500000a03012c_UUID:0/0 lens 368/264 e 0 to 0 dl 1431853795 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:08:15 ALPL202 kernel: LustreError: 7351:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2774 previous similar messages May 17 17:08:20 ALPL202 kernel: Lustre: 6437:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 17 17:08:20 ALPL202 kernel: Lustre: 6437:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2766 previous similar messages May 17 17:09:12 ALPL202 kernel: Lustre: 7311:0:(service.c:812:ptlrpc_at_send_early_reply()) @@@ Couldn't add any time (5/-207), not sending early reply May 17 17:09:12 ALPL202 kernel: req@ffff8105e1458800 x1499014177743406/t0 o101->9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@NET_0x500000a03012f_UUID:0/0 lens 296/0 e 5 to 0 dl 1431853757 ref 2 fl Interpret:/0/0 rc 0/0 May 17 17:18:09 ALPL202 kernel: Lustre: 7591:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: fc0811da-8123-3956-a1b8-e1d99a941c14 reconnecting May 17 17:18:09 ALPL202 kernel: Lustre: 7591:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2822 previous similar messages May 17 17:18:16 ALPL202 kernel: LustreError: 6398:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810078de4800 x1499649574371008/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431854396 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:18:16 ALPL202 kernel: LustreError: 6398:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2815 previous similar messages May 17 17:18:20 ALPL202 kernel: Lustre: 7649:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@10.3.1.47@o2ib to 0xffff81021edfdc00; still busy with 1 active RPCs May 17 17:18:20 ALPL202 kernel: Lustre: 7649:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2817 previous similar messages May 17 17:28:09 ALPL202 kernel: Lustre: 7629:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 17:28:09 ALPL202 kernel: Lustre: 7629:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2829 previous similar messages May 17 17:28:16 ALPL202 kernel: LustreError: 7654:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102aa05b050 x1499645795848848/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431854996 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:28:16 ALPL202 kernel: LustreError: 7654:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 17:28:20 ALPL202 kernel: Lustre: 7531:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from a1950f92-7e5d-7d7b-af8e-ae39c24d40ed@10.3.1.21@o2ib to 0xffff810165d97e00; still busy with 2 active RPCs May 17 17:28:20 ALPL202 kernel: Lustre: 7531:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 17:38:10 ALPL202 kernel: Lustre: 6502:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f32cf79c-e087-bfe3-6e8f-7fde95ab168b reconnecting May 17 17:38:10 ALPL202 kernel: Lustre: 6502:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 17 17:38:16 ALPL202 kernel: LustreError: 7571:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e4711800 x1499008110776758/t0 o8->17e28ade-58ed-211a-39c9-291989587703@:0/0 lens 368/264 e 0 to 0 dl 1431855596 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:38:16 ALPL202 kernel: LustreError: 7571:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2843 previous similar messages May 17 17:38:21 ALPL202 kernel: Lustre: 7285:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d7a5a345-9841-f534-ddde-a2808cdd637c@10.3.1.34@o2ib to 0xffff81061f5f1600; still busy with 3 active RPCs May 17 17:38:21 ALPL202 kernel: Lustre: 7285:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2829 previous similar messages May 17 17:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 17:48:10 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 22492cf0-9baa-5c7d-4f94-ca5f8ab20776 reconnecting May 17 17:48:10 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 17 17:48:16 ALPL202 kernel: LustreError: 7278:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105fc4da400 x1499682232922826/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431856196 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:48:16 ALPL202 kernel: LustreError: 7278:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2829 previous similar messages May 17 17:48:21 ALPL202 kernel: Lustre: 6474:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f32cf79c-e087-bfe3-6e8f-7fde95ab168b@10.3.1.15@o2ib to 0xffff8105c33c4e00; still busy with 3 active RPCs May 17 17:48:21 ALPL202 kernel: Lustre: 6474:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2841 previous similar messages May 17 17:58:10 ALPL202 kernel: Lustre: 7306:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d7a5a345-9841-f534-ddde-a2808cdd637c reconnecting May 17 17:58:10 ALPL202 kernel: Lustre: 7306:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2827 previous similar messages May 17 17:58:17 ALPL202 kernel: LustreError: 7625:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106224b6400 x1499597025752529/t0 o8->f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@NET_0x500000a030116_UUID:0/0 lens 368/264 e 0 to 0 dl 1431856797 ref 1 fl Interpret:/0/0 rc -16/0 May 17 17:58:17 ALPL202 kernel: LustreError: 7625:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 17 17:58:22 ALPL202 kernel: Lustre: 7330:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04cfad47-91cb-7688-6409-2530587af605@10.3.1.48@o2ib to 0xffff810479a1a600; still busy with 2 active RPCs May 17 17:58:22 ALPL202 kernel: Lustre: 7330:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 18:08:10 ALPL202 kernel: Lustre: 7300:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 17 18:08:10 ALPL202 kernel: Lustre: 7300:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2840 previous similar messages May 17 18:08:17 ALPL202 kernel: LustreError: 7337:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81062261ac00 x1499109800622542/t0 o8->22492cf0-9baa-5c7d-4f94-ca5f8ab20776@NET_0x500000a030125_UUID:0/0 lens 368/264 e 0 to 0 dl 1431857397 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:08:17 ALPL202 kernel: LustreError: 7337:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 18:08:22 ALPL202 kernel: Lustre: 6514:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from de0fa4c5-b63e-b22c-6d6a-d2383f854657@10.3.1.20@o2ib to 0xffff810192864200; still busy with 4 active RPCs May 17 18:08:22 ALPL202 kernel: Lustre: 6514:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 17 18:18:10 ALPL202 kernel: Lustre: 7376:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04cfad47-91cb-7688-6409-2530587af605 reconnecting May 17 18:18:10 ALPL202 kernel: Lustre: 7376:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 17 18:18:17 ALPL202 kernel: LustreError: 7384:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81035e4bb800 x1499649574512407/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431857997 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:18:17 ALPL202 kernel: LustreError: 7384:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 17 18:18:22 ALPL202 kernel: Lustre: 7348:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 17 18:18:22 ALPL202 kernel: Lustre: 7348:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2841 previous similar messages May 17 18:28:10 ALPL202 kernel: Lustre: 7438:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 7b1b193e-71f6-a13d-fc5a-cd188b052138 reconnecting May 17 18:28:10 ALPL202 kernel: Lustre: 7438:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 17 18:28:17 ALPL202 kernel: LustreError: 7303:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101914f2450 x1499839927947417/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431858597 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:28:17 ALPL202 kernel: LustreError: 7303:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2842 previous similar messages May 17 18:28:22 ALPL202 kernel: Lustre: 6394:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 17 18:28:22 ALPL202 kernel: Lustre: 6394:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 17 18:38:10 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: a1950f92-7e5d-7d7b-af8e-ae39c24d40ed reconnecting May 17 18:38:10 ALPL202 kernel: Lustre: 7351:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 17 18:38:17 ALPL202 kernel: LustreError: 7348:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105d0d6f800 x1499008110916839/t0 o8->17e28ade-58ed-211a-39c9-291989587703@:0/0 lens 368/264 e 0 to 0 dl 1431859197 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:38:17 ALPL202 kernel: LustreError: 7348:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 18:38:22 ALPL202 kernel: Lustre: 7432:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f32cf79c-e087-bfe3-6e8f-7fde95ab168b@10.3.1.15@o2ib to 0xffff8105c33c4e00; still busy with 3 active RPCs May 17 18:38:22 ALPL202 kernel: Lustre: 7432:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 18:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 18:48:10 ALPL202 kernel: Lustre: 7456:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4996fcc3-85b7-659a-e4a1-572255254f65 reconnecting May 17 18:48:10 ALPL202 kernel: Lustre: 7456:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2828 previous similar messages May 17 18:48:18 ALPL202 kernel: LustreError: 7480:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810275378c00 x1499649574582534/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431859798 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:48:18 ALPL202 kernel: LustreError: 7480:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 18:48:23 ALPL202 kernel: Lustre: 7334:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@10.3.1.22@o2ib to 0xffff81063fe3a400; still busy with 1 active RPCs May 17 18:48:23 ALPL202 kernel: Lustre: 7334:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 18:58:10 ALPL202 kernel: Lustre: 7379:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 72657e2e-01a5-e000-c945-6edb35b026e6 reconnecting May 17 18:58:10 ALPL202 kernel: Lustre: 7379:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 17 18:58:18 ALPL202 kernel: LustreError: 7475:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810291d3f000 x1499006744264122/t0 o8->ada59592-d805-c261-0c63-be0a8c2fa922@NET_0x500000a03011f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431860398 ref 1 fl Interpret:/0/0 rc -16/0 May 17 18:58:18 ALPL202 kernel: LustreError: 7475:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 17 18:58:23 ALPL202 kernel: Lustre: 7322:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d7a5a345-9841-f534-ddde-a2808cdd637c@10.3.1.34@o2ib to 0xffff81061f5f1600; still busy with 3 active RPCs May 17 18:58:23 ALPL202 kernel: Lustre: 7322:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 17 19:08:10 ALPL202 kernel: Lustre: 6423:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e8917316-444f-9483-db5a-7c7fb1549a1e reconnecting May 17 19:08:10 ALPL202 kernel: Lustre: 6423:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2834 previous similar messages May 17 19:08:18 ALPL202 kernel: LustreError: 7506:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102ae416400 x1499008110986539/t0 o8->17e28ade-58ed-211a-39c9-291989587703@:0/0 lens 368/264 e 0 to 0 dl 1431860998 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:08:18 ALPL202 kernel: LustreError: 7506:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 17 19:08:23 ALPL202 kernel: Lustre: 7437:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from b723915e-bd6d-5de8-3f38-d333eb1db2ed@10.3.1.3@o2ib to 0xffff81036846c200; still busy with 2 active RPCs May 17 19:08:23 ALPL202 kernel: Lustre: 7437:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 17 19:18:11 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f0bae738-4650-c254-ee38-ca124e1eea57 reconnecting May 17 19:18:11 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 17 19:18:18 ALPL202 kernel: LustreError: 7596:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102aa518400 x1499014177876668/t0 o8->9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@NET_0x500000a03012f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431861598 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:18:18 ALPL202 kernel: LustreError: 7596:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 17 19:18:23 ALPL202 kernel: Lustre: 7589:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 86d0ab2a-246f-a784-2744-c50f68cee379@10.3.1.26@o2ib to 0xffff810192864800; still busy with 2 active RPCs May 17 19:18:23 ALPL202 kernel: Lustre: 7589:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 17 19:28:11 ALPL202 kernel: Lustre: 6413:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 17 19:28:11 ALPL202 kernel: Lustre: 6413:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 19:28:19 ALPL202 kernel: LustreError: 7395:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103518c4850 x1497853923030779/t0 o8->efebfe6f-6429-bcd2-fd37-e715eaf8983b@NET_0x500000a030124_UUID:0/0 lens 368/264 e 0 to 0 dl 1431862199 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:28:19 ALPL202 kernel: LustreError: 7395:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 17 19:28:23 ALPL202 kernel: Lustre: 7303:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4996fcc3-85b7-659a-e4a1-572255254f65@10.3.1.25@o2ib to 0xffff8105c33c4400; still busy with 3 active RPCs May 17 19:28:23 ALPL202 kernel: Lustre: 7303:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 17 19:38:11 ALPL202 kernel: Lustre: 6397:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 86d0ab2a-246f-a784-2744-c50f68cee379 reconnecting May 17 19:38:11 ALPL202 kernel: Lustre: 6397:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 19:38:19 ALPL202 kernel: LustreError: 7555:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e1553c00 x1497754014953537/t0 o8->4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@NET_0x500000a030111_UUID:0/0 lens 368/264 e 0 to 0 dl 1431862799 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:38:19 ALPL202 kernel: LustreError: 7555:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 19:38:23 ALPL202 kernel: Lustre: 7502:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@10.3.1.43@o2ib to 0xffff81063fe3aa00; still busy with 4 active RPCs May 17 19:38:23 ALPL202 kernel: Lustre: 7502:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2842 previous similar messages May 17 19:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 19:48:11 ALPL202 kernel: Lustre: 7416:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84 reconnecting May 17 19:48:11 ALPL202 kernel: Lustre: 7416:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2834 previous similar messages May 17 19:48:19 ALPL202 kernel: LustreError: 7326:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810163793c00 x1499839927957357/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431863399 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:48:19 ALPL202 kernel: LustreError: 7326:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 19:48:23 ALPL202 kernel: Lustre: 6396:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 929aa131-7e43-4b34-d056-6e36a0b13e1a@10.3.1.1@o2ib to 0xffff8105c33c4600; still busy with 3 active RPCs May 17 19:48:23 ALPL202 kernel: Lustre: 6396:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 17 19:58:11 ALPL202 kernel: Lustre: 7473:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 19:58:11 ALPL202 kernel: Lustre: 7473:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 19:58:19 ALPL202 kernel: LustreError: 7528:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101bdb0f400 x1499033021225545/t0 o8->79a59854-31b9-d753-4e22-8912905a5085@:0/0 lens 368/264 e 0 to 0 dl 1431863999 ref 1 fl Interpret:/0/0 rc -16/0 May 17 19:58:19 ALPL202 kernel: LustreError: 7528:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 19:58:24 ALPL202 kernel: Lustre: 6431:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f0bae738-4650-c254-ee38-ca124e1eea57@10.3.1.11@o2ib to 0xffff810517df7e00; still busy with 4 active RPCs May 17 19:58:24 ALPL202 kernel: Lustre: 6431:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 17 20:08:11 ALPL202 kernel: Lustre: 7504:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: fc0811da-8123-3956-a1b8-e1d99a941c14 reconnecting May 17 20:08:11 ALPL202 kernel: Lustre: 7504:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 20:08:19 ALPL202 kernel: LustreError: 6426:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101dc7bf800 x1485800370868846/t0 o8->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431864599 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:08:19 ALPL202 kernel: LustreError: 6426:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 17 20:08:24 ALPL202 kernel: Lustre: 7393:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04cfad47-91cb-7688-6409-2530587af605@10.3.1.48@o2ib to 0xffff810479a1a600; still busy with 2 active RPCs May 17 20:08:24 ALPL202 kernel: Lustre: 7393:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 20:18:12 ALPL202 kernel: Lustre: 7319:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 17 20:18:12 ALPL202 kernel: Lustre: 7319:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 20:18:20 ALPL202 kernel: LustreError: 7649:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104da253800 x1485802398320487/t0 o8->72657e2e-01a5-e000-c945-6edb35b026e6@NET_0x500000a03012a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431865200 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:18:20 ALPL202 kernel: LustreError: 7649:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2839 previous similar messages May 17 20:18:24 ALPL202 kernel: Lustre: 7477:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ea79a342-578d-a338-caef-c51d0fe5e904@10.3.1.46@o2ib to 0xffff810372190200; still busy with 2 active RPCs May 17 20:18:24 ALPL202 kernel: Lustre: 7477:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 20:28:12 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 9036737a-5bd8-c797-d654-783c8fbb4961 reconnecting May 17 20:28:12 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 17 20:28:20 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105ad756400 x1497757445308418/t0 o8->13307ac7-2a42-d2e1-9f17-7cd2822f32c8@NET_0x500000a030129_UUID:0/0 lens 368/264 e 0 to 0 dl 1431865800 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:28:20 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 17 20:28:24 ALPL202 kernel: Lustre: 6470:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 17 20:28:24 ALPL202 kernel: Lustre: 6470:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 17 20:38:12 ALPL202 kernel: Lustre: 7396:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 17 20:38:12 ALPL202 kernel: Lustre: 7396:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 17 20:38:20 ALPL202 kernel: LustreError: 6498:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105a9b88800 x1499689913157665/t0 o8->ade1e689-13ba-5165-30df-204eabc4dae8@NET_0x500000a030112_UUID:0/0 lens 368/264 e 0 to 0 dl 1431866400 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:38:20 ALPL202 kernel: LustreError: 6498:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 20:38:24 ALPL202 kernel: Lustre: 7519:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17e28ade-58ed-211a-39c9-291989587703@10.3.1.29@o2ib to 0xffff81016379be00; still busy with 2 active RPCs May 17 20:38:24 ALPL202 kernel: Lustre: 7519:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 20:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 20:48:12 ALPL202 kernel: Lustre: 7564:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f9fdf5c1-f5d4-d701-81d8-f6afd06169ef reconnecting May 17 20:48:12 ALPL202 kernel: Lustre: 7564:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 20:48:20 ALPL202 kernel: LustreError: 7522:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101d3dea400 x1499006744546024/t0 o8->ada59592-d805-c261-0c63-be0a8c2fa922@NET_0x500000a03011f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431867000 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:48:20 ALPL202 kernel: LustreError: 7522:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 17 20:48:24 ALPL202 kernel: Lustre: 7475:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 17 20:48:24 ALPL202 kernel: Lustre: 7475:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 17 20:58:12 ALPL202 kernel: Lustre: 7556:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e3633471-5b20-ed6f-3476-b9c3d5d78060 reconnecting May 17 20:58:12 ALPL202 kernel: Lustre: 7556:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 17 20:58:20 ALPL202 kernel: LustreError: 6509:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105dc2b4000 x1499017964608592/t0 o8->1ed5a549-efbb-515c-ff9a-8b8741af8256@NET_0x500000a030127_UUID:0/0 lens 368/264 e 0 to 0 dl 1431867600 ref 1 fl Interpret:/0/0 rc -16/0 May 17 20:58:20 ALPL202 kernel: LustreError: 6509:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 20:58:25 ALPL202 kernel: Lustre: 7372:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from efebfe6f-6429-bcd2-fd37-e715eaf8983b@10.3.1.36@o2ib to 0xffff81065516f800; still busy with 4 active RPCs May 17 20:58:25 ALPL202 kernel: Lustre: 7372:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 17 21:08:12 ALPL202 kernel: Lustre: 7516:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d reconnecting May 17 21:08:12 ALPL202 kernel: Lustre: 7516:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 21:08:21 ALPL202 kernel: LustreError: 7494:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104ba8cfc00 x1499109801068012/t0 o8->22492cf0-9baa-5c7d-4f94-ca5f8ab20776@NET_0x500000a030125_UUID:0/0 lens 368/264 e 0 to 0 dl 1431868201 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:08:21 ALPL202 kernel: LustreError: 7494:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 17 21:08:25 ALPL202 kernel: Lustre: 6498:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 2a3fd8ac-ad9e-6c5c-eb5a-e9e64491e0ac@10.3.1.40@o2ib to 0xffff8105d807fa00; still busy with 1 active RPCs May 17 21:08:25 ALPL202 kernel: Lustre: 6498:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 21:18:12 ALPL202 kernel: Lustre: 7545:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 929aa131-7e43-4b34-d056-6e36a0b13e1a reconnecting May 17 21:18:12 ALPL202 kernel: Lustre: 7545:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 17 21:18:21 ALPL202 kernel: LustreError: 6472:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103be687800 x1499018007891281/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431868801 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:18:21 ALPL202 kernel: LustreError: 6472:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 17 21:18:25 ALPL202 kernel: Lustre: 7516:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e986becc-307e-f48d-480a-444f9c6b233c@10.3.1.35@o2ib to 0xffff81017d1a0600; still busy with 4 active RPCs May 17 21:18:25 ALPL202 kernel: Lustre: 7516:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 17 21:28:13 ALPL202 kernel: Lustre: 7609:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4996fcc3-85b7-659a-e4a1-572255254f65 reconnecting May 17 21:28:13 ALPL202 kernel: Lustre: 7609:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2839 previous similar messages May 17 21:28:21 ALPL202 kernel: LustreError: 7565:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101c3623800 x1499653316483477/t0 o8->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 368/264 e 0 to 0 dl 1431869401 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:28:21 ALPL202 kernel: LustreError: 7565:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 21:28:25 ALPL202 kernel: Lustre: 7294:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 17 21:28:25 ALPL202 kernel: Lustre: 7294:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 17 21:38:13 ALPL202 kernel: Lustre: 7557:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d reconnecting May 17 21:38:13 ALPL202 kernel: Lustre: 7557:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 21:38:21 ALPL202 kernel: LustreError: 7429:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81045288c800 x1499645796364700/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431870001 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:38:21 ALPL202 kernel: LustreError: 7429:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 17 21:38:25 ALPL202 kernel: Lustre: 6472:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4996fcc3-85b7-659a-e4a1-572255254f65@10.3.1.25@o2ib to 0xffff8105c33c4400; still busy with 3 active RPCs May 17 21:38:25 ALPL202 kernel: Lustre: 6472:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 17 21:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 21:48:14 ALPL202 kernel: Lustre: 6513:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f9ed569f-59c6-0f68-7b91-ef32fbaef2d8 reconnecting May 17 21:48:14 ALPL202 kernel: Lustre: 6513:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 17 21:48:21 ALPL202 kernel: LustreError: 7362:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81006973e000 x1499018007965715/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431870601 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:48:21 ALPL202 kernel: LustreError: 7362:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 17 21:48:26 ALPL202 kernel: Lustre: 7300:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 72657e2e-01a5-e000-c945-6edb35b026e6@10.3.1.42@o2ib to 0xffff8103c7e72a00; still busy with 1 active RPCs May 17 21:48:26 ALPL202 kernel: Lustre: 7300:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 21:58:14 ALPL202 kernel: Lustre: 7414:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: b723915e-bd6d-5de8-3f38-d333eb1db2ed reconnecting May 17 21:58:14 ALPL202 kernel: Lustre: 7414:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 21:58:21 ALPL202 kernel: LustreError: 7635:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101be1d1000 x1499649575029980/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431871201 ref 1 fl Interpret:/0/0 rc -16/0 May 17 21:58:21 ALPL202 kernel: LustreError: 7635:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2831 previous similar messages May 17 21:58:26 ALPL202 kernel: Lustre: 7655:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 17 21:58:26 ALPL202 kernel: Lustre: 7655:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 22:08:14 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 22492cf0-9baa-5c7d-4f94-ca5f8ab20776 reconnecting May 17 22:08:14 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2830 previous similar messages May 17 22:08:21 ALPL202 kernel: LustreError: 7392:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810356041000 x1499034337612113/t0 o8->7b1b193e-71f6-a13d-fc5a-cd188b052138@NET_0x500000a030126_UUID:0/0 lens 368/264 e 0 to 0 dl 1431871801 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:08:21 ALPL202 kernel: LustreError: 7392:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 17 22:08:26 ALPL202 kernel: Lustre: 7423:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4996fcc3-85b7-659a-e4a1-572255254f65@10.3.1.25@o2ib to 0xffff8105c33c4400; still busy with 3 active RPCs May 17 22:08:26 ALPL202 kernel: Lustre: 7423:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 17 22:18:14 ALPL202 kernel: Lustre: 7278:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 17 22:18:14 ALPL202 kernel: Lustre: 7278:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 17 22:18:21 ALPL202 kernel: LustreError: 6474:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103fde5e800 x1499839927975487/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431872401 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:18:21 ALPL202 kernel: LustreError: 6474:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 17 22:18:26 ALPL202 kernel: Lustre: 7325:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84@10.3.1.24@o2ib to 0xffff810ab8f72c00; still busy with 1 active RPCs May 17 22:18:26 ALPL202 kernel: Lustre: 7325:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 17 22:28:14 ALPL202 kernel: Lustre: 7382:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e986becc-307e-f48d-480a-444f9c6b233c reconnecting May 17 22:28:14 ALPL202 kernel: Lustre: 7382:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2832 previous similar messages May 17 22:28:22 ALPL202 kernel: LustreError: 7348:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81051531d000 x1499682233571948/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431873002 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:28:22 ALPL202 kernel: LustreError: 7348:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2830 previous similar messages May 17 22:28:26 ALPL202 kernel: Lustre: 6407:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17e28ade-58ed-211a-39c9-291989587703@10.3.1.29@o2ib to 0xffff81016379be00; still busy with 2 active RPCs May 17 22:28:26 ALPL202 kernel: Lustre: 6407:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 22:38:14 ALPL202 kernel: Lustre: 7659:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84 reconnecting May 17 22:38:14 ALPL202 kernel: Lustre: 7659:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 22:38:22 ALPL202 kernel: LustreError: 7556:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81033ecccc00 x1497757445617404/t0 o8->13307ac7-2a42-d2e1-9f17-7cd2822f32c8@NET_0x500000a030129_UUID:0/0 lens 368/264 e 0 to 0 dl 1431873602 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:38:22 ALPL202 kernel: LustreError: 7556:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2841 previous similar messages May 17 22:38:26 ALPL202 kernel: Lustre: 7514:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 17 22:38:26 ALPL202 kernel: Lustre: 7514:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 17 22:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 22:48:14 ALPL202 kernel: Lustre: 6475:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 79a59854-31b9-d753-4e22-8912905a5085 reconnecting May 17 22:48:14 ALPL202 kernel: Lustre: 6475:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 22:48:22 ALPL202 kernel: LustreError: 7292:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e1553400 x1498183791100620/t0 o8->4996fcc3-85b7-659a-e4a1-572255254f65@NET_0x500000a030119_UUID:0/0 lens 368/264 e 0 to 0 dl 1431874202 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:48:22 ALPL202 kernel: LustreError: 7292:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 22:48:26 ALPL202 kernel: Lustre: 7308:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 929aa131-7e43-4b34-d056-6e36a0b13e1a@10.3.1.1@o2ib to 0xffff8105c33c4600; still busy with 3 active RPCs May 17 22:48:26 ALPL202 kernel: Lustre: 7308:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 17 22:58:15 ALPL202 kernel: Lustre: 7338:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 22492cf0-9baa-5c7d-4f94-ca5f8ab20776 reconnecting May 17 22:58:15 ALPL202 kernel: Lustre: 7338:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 17 22:58:23 ALPL202 kernel: LustreError: 7437:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102f910ac00 x1485800371175842/t0 o8->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431874803 ref 1 fl Interpret:/0/0 rc -16/0 May 17 22:58:23 ALPL202 kernel: LustreError: 7437:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 22:58:26 ALPL202 kernel: Lustre: 7399:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@10.3.1.47@o2ib to 0xffff81021edfdc00; still busy with 1 active RPCs May 17 22:58:26 ALPL202 kernel: Lustre: 7399:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 17 23:08:15 ALPL202 kernel: Lustre: 7648:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: de0fa4c5-b63e-b22c-6d6a-d2383f854657 reconnecting May 17 23:08:15 ALPL202 kernel: Lustre: 7648:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 23:08:23 ALPL202 kernel: LustreError: 7430:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104109df050 x1485800793754227/t0 o8->dee84236-56bc-f2f7-cd2c-ba3d555fd617@NET_0x500000a03011b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431875403 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:08:23 ALPL202 kernel: LustreError: 7430:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 23:08:27 ALPL202 kernel: Lustre: 7301:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f32cf79c-e087-bfe3-6e8f-7fde95ab168b@10.3.1.15@o2ib to 0xffff8105c33c4e00; still busy with 3 active RPCs May 17 23:08:27 ALPL202 kernel: Lustre: 7301:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 17 23:18:15 ALPL202 kernel: Lustre: 7507:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 86d0ab2a-246f-a784-2744-c50f68cee379 reconnecting May 17 23:18:15 ALPL202 kernel: Lustre: 7507:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 17 23:18:23 ALPL202 kernel: LustreError: 6411:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102e0548c00 x1499597079971473/t0 o8->f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@NET_0x500000a030116_UUID:0/0 lens 368/264 e 0 to 0 dl 1431876003 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:18:23 ALPL202 kernel: LustreError: 6411:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 17 23:18:27 ALPL202 kernel: Lustre: 7652:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from dee84236-56bc-f2f7-cd2c-ba3d555fd617@10.3.1.27@o2ib to 0xffff81050f727a00; still busy with 2 active RPCs May 17 23:18:27 ALPL202 kernel: Lustre: 7652:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 17 23:28:15 ALPL202 kernel: Lustre: 7452:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ade1e689-13ba-5165-30df-204eabc4dae8 reconnecting May 17 23:28:15 ALPL202 kernel: Lustre: 7452:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 17 23:28:23 ALPL202 kernel: LustreError: 7385:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102c0b7e050 x1498234009438862/t0 o8->ea79a342-578d-a338-caef-c51d0fe5e904@NET_0x500000a03012e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431876603 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:28:23 ALPL202 kernel: LustreError: 7385:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 17 23:28:27 ALPL202 kernel: Lustre: 7346:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@10.3.1.22@o2ib to 0xffff81063fe3a400; still busy with 1 active RPCs May 17 23:28:27 ALPL202 kernel: Lustre: 7346:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 17 23:38:15 ALPL202 kernel: Lustre: 6524:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d3e32527-f91f-d191-fbe6-fbc0e0ba6beb reconnecting May 17 23:38:15 ALPL202 kernel: Lustre: 6524:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 17 23:38:23 ALPL202 kernel: LustreError: 7450:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810573d8d400 x1499839927985140/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431877203 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:38:23 ALPL202 kernel: LustreError: 7450:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 23:38:27 ALPL202 kernel: Lustre: 7562:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d7a5a345-9841-f534-ddde-a2808cdd637c@10.3.1.34@o2ib to 0xffff81061f5f1600; still busy with 3 active RPCs May 17 23:38:27 ALPL202 kernel: Lustre: 7562:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 17 23:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 17 23:48:15 ALPL202 kernel: Lustre: 7458:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f32cf79c-e087-bfe3-6e8f-7fde95ab168b reconnecting May 17 23:48:15 ALPL202 kernel: Lustre: 7458:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 17 23:48:23 ALPL202 kernel: LustreError: 6439:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81062252e400 x1498527208821636/t0 o8->fc0811da-8123-3956-a1b8-e1d99a941c14@NET_0x500000a03011c_UUID:0/0 lens 368/264 e 0 to 0 dl 1431877803 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:48:23 ALPL202 kernel: LustreError: 6439:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 17 23:48:27 ALPL202 kernel: Lustre: 7595:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04ec28a8-eb95-c793-1a9c-7827d08a2f27@10.3.1.9@o2ib to 0xffff81079778a600; still busy with 4 active RPCs May 17 23:48:27 ALPL202 kernel: Lustre: 7595:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 17 23:58:16 ALPL202 kernel: Lustre: 7410:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: de0fa4c5-b63e-b22c-6d6a-d2383f854657 reconnecting May 17 23:58:16 ALPL202 kernel: Lustre: 7410:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 17 23:58:24 ALPL202 kernel: LustreError: 7408:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81009864d800 x1498234009495189/t0 o8->ea79a342-578d-a338-caef-c51d0fe5e904@NET_0x500000a03012e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431878404 ref 1 fl Interpret:/0/0 rc -16/0 May 17 23:58:24 ALPL202 kernel: LustreError: 7408:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 17 23:58:27 ALPL202 kernel: Lustre: 6453:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from fc0811da-8123-3956-a1b8-e1d99a941c14@10.3.1.28@o2ib to 0xffff8105d807f000; still busy with 4 active RPCs May 17 23:58:27 ALPL202 kernel: Lustre: 6453:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2839 previous similar messages May 18 00:08:16 ALPL202 kernel: Lustre: 7554:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 18 00:08:16 ALPL202 kernel: Lustre: 7554:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 18 00:08:24 ALPL202 kernel: LustreError: 7546:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e5918050 x1487072295289652/t0 o8->86d0ab2a-246f-a784-2744-c50f68cee379@NET_0x500000a03011a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431879004 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:08:24 ALPL202 kernel: LustreError: 7546:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2841 previous similar messages May 18 00:08:27 ALPL202 kernel: Lustre: 6476:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ada59592-d805-c261-0c63-be0a8c2fa922@10.3.1.31@o2ib to 0xffff810372190a00; still busy with 3 active RPCs May 18 00:08:27 ALPL202 kernel: Lustre: 6476:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 18 00:18:16 ALPL202 kernel: Lustre: 7555:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e986becc-307e-f48d-480a-444f9c6b233c reconnecting May 18 00:18:16 ALPL202 kernel: Lustre: 7555:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 18 00:18:24 ALPL202 kernel: LustreError: 7431:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103354d8800 x1487068467493985/t0 o8->2a3fd8ac-ad9e-6c5c-eb5a-e9e64491e0ac@NET_0x500000a030128_UUID:0/0 lens 368/264 e 0 to 0 dl 1431879604 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:18:24 ALPL202 kernel: LustreError: 7431:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2839 previous similar messages May 18 00:18:27 ALPL202 kernel: Lustre: 7586:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17e28ade-58ed-211a-39c9-291989587703@10.3.1.29@o2ib to 0xffff81016379be00; still busy with 2 active RPCs May 18 00:18:27 ALPL202 kernel: Lustre: 7586:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 00:28:16 ALPL202 kernel: Lustre: 6417:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 18 00:28:16 ALPL202 kernel: Lustre: 6417:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 00:28:24 ALPL202 kernel: LustreError: 7598:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101639c6800 x1499033021688455/t0 o8->79a59854-31b9-d753-4e22-8912905a5085@:0/0 lens 368/264 e 0 to 0 dl 1431880204 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:28:24 ALPL202 kernel: LustreError: 7598:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2827 previous similar messages May 18 00:28:28 ALPL202 kernel: Lustre: 6488:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 22492cf0-9baa-5c7d-4f94-ca5f8ab20776@10.3.1.37@o2ib to 0xffff810372190800; still busy with 4 active RPCs May 18 00:28:28 ALPL202 kernel: Lustre: 6488:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 00:38:16 ALPL202 kernel: Lustre: 7633:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 17e28ade-58ed-211a-39c9-291989587703 reconnecting May 18 00:38:16 ALPL202 kernel: Lustre: 7633:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 00:38:25 ALPL202 kernel: LustreError: 7549:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810286105850 x1486456503672083/t0 o8->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 368/264 e 0 to 0 dl 1431880805 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:38:25 ALPL202 kernel: LustreError: 7549:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 18 00:38:28 ALPL202 kernel: Lustre: 7307:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 18 00:38:28 ALPL202 kernel: Lustre: 7307:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 18 00:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 00:48:16 ALPL202 kernel: Lustre: 7280:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 17342f99-c92f-d6af-0247-fdddc176fe9b reconnecting May 18 00:48:16 ALPL202 kernel: Lustre: 7280:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2839 previous similar messages May 18 00:48:25 ALPL202 kernel: LustreError: 6466:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104862c2400 x1499689913749629/t0 o8->ade1e689-13ba-5165-30df-204eabc4dae8@NET_0x500000a030112_UUID:0/0 lens 368/264 e 0 to 0 dl 1431881405 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:48:25 ALPL202 kernel: LustreError: 6466:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2841 previous similar messages May 18 00:48:28 ALPL202 kernel: Lustre: 7491:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e986becc-307e-f48d-480a-444f9c6b233c@10.3.1.35@o2ib to 0xffff81017d1a0600; still busy with 4 active RPCs May 18 00:48:28 ALPL202 kernel: Lustre: 7491:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 00:58:16 ALPL202 kernel: Lustre: 7656:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 17e28ade-58ed-211a-39c9-291989587703 reconnecting May 18 00:58:16 ALPL202 kernel: Lustre: 7656:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2830 previous similar messages May 18 00:58:25 ALPL202 kernel: LustreError: 7562:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102aa518400 x1499008111800303/t0 o8->17e28ade-58ed-211a-39c9-291989587703@:0/0 lens 368/264 e 0 to 0 dl 1431882005 ref 1 fl Interpret:/0/0 rc -16/0 May 18 00:58:25 ALPL202 kernel: LustreError: 7562:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 18 00:58:28 ALPL202 kernel: Lustre: 7510:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84@10.3.1.24@o2ib to 0xffff810ab8f72c00; still busy with 1 active RPCs May 18 00:58:28 ALPL202 kernel: Lustre: 7510:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 01:08:16 ALPL202 kernel: Lustre: 7504:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e8917316-444f-9483-db5a-7c7fb1549a1e reconnecting May 18 01:08:16 ALPL202 kernel: Lustre: 7504:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2840 previous similar messages May 18 01:08:25 ALPL202 kernel: LustreError: 7552:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105d9111400 x1499006652670014/t0 o8->e8917316-444f-9483-db5a-7c7fb1549a1e@NET_0x500000a03011e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431882605 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:08:25 ALPL202 kernel: LustreError: 7552:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 01:08:28 ALPL202 kernel: Lustre: 6411:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 1ed5a549-efbb-515c-ff9a-8b8741af8256@10.3.1.39@o2ib to 0xffff81050f727600; still busy with 1 active RPCs May 18 01:08:28 ALPL202 kernel: Lustre: 6411:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 18 01:18:17 ALPL202 kernel: Lustre: 6505:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04cfad47-91cb-7688-6409-2530587af605 reconnecting May 18 01:18:17 ALPL202 kernel: Lustre: 6505:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 01:18:26 ALPL202 kernel: LustreError: 7416:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101d1509c00 x1499018008486922/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431883206 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:18:26 ALPL202 kernel: LustreError: 7416:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2830 previous similar messages May 18 01:18:29 ALPL202 kernel: Lustre: 7605:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 18 01:18:29 ALPL202 kernel: Lustre: 7605:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2840 previous similar messages May 18 01:28:17 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 9036737a-5bd8-c797-d654-783c8fbb4961 reconnecting May 18 01:28:17 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 01:28:26 ALPL202 kernel: LustreError: 6525:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8102903dcc00 x1497753931690652/t0 o8->9036737a-5bd8-c797-d654-783c8fbb4961@NET_0x500000a030106_UUID:0/0 lens 368/264 e 0 to 0 dl 1431883806 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:28:26 ALPL202 kernel: LustreError: 6525:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 01:28:29 ALPL202 kernel: Lustre: 7431:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f0bae738-4650-c254-ee38-ca124e1eea57@10.3.1.11@o2ib to 0xffff810517df7e00; still busy with 4 active RPCs May 18 01:28:29 ALPL202 kernel: Lustre: 7431:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 18 01:38:17 ALPL202 kernel: Lustre: 6530:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ade1e689-13ba-5165-30df-204eabc4dae8 reconnecting May 18 01:38:17 ALPL202 kernel: Lustre: 6530:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2839 previous similar messages May 18 01:38:26 ALPL202 kernel: LustreError: 7537:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105759fe850 x1499689913867511/t0 o8->ade1e689-13ba-5165-30df-204eabc4dae8@NET_0x500000a030112_UUID:0/0 lens 368/264 e 0 to 0 dl 1431884406 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:38:26 ALPL202 kernel: LustreError: 7537:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 01:38:29 ALPL202 kernel: Lustre: 7298:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 9036737a-5bd8-c797-d654-783c8fbb4961@10.3.1.6@o2ib to 0xffff8105c33c4800; still busy with 1 active RPCs May 18 01:38:29 ALPL202 kernel: Lustre: 7298:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 01:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 01:48:17 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 17e28ade-58ed-211a-39c9-291989587703 reconnecting May 18 01:48:17 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 01:48:26 ALPL202 kernel: LustreError: 7432:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81059d5fd000 x1499008111916663/t0 o8->17e28ade-58ed-211a-39c9-291989587703@:0/0 lens 368/264 e 0 to 0 dl 1431885006 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:48:26 ALPL202 kernel: LustreError: 7432:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2831 previous similar messages May 18 01:48:29 ALPL202 kernel: Lustre: 7506:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 18 01:48:29 ALPL202 kernel: Lustre: 7506:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 01:58:17 ALPL202 kernel: Lustre: 7593:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 79a59854-31b9-d753-4e22-8912905a5085 reconnecting May 18 01:58:17 ALPL202 kernel: Lustre: 7593:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2837 previous similar messages May 18 01:58:26 ALPL202 kernel: LustreError: 7458:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81042e93bc00 x1499033021841800/t0 o8->79a59854-31b9-d753-4e22-8912905a5085@:0/0 lens 368/264 e 0 to 0 dl 1431885606 ref 1 fl Interpret:/0/0 rc -16/0 May 18 01:58:26 ALPL202 kernel: LustreError: 7458:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 01:58:29 ALPL202 kernel: Lustre: 7491:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84@10.3.1.24@o2ib to 0xffff810ab8f72c00; still busy with 1 active RPCs May 18 01:58:29 ALPL202 kernel: Lustre: 7491:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 02:08:17 ALPL202 kernel: Lustre: 7394:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 929aa131-7e43-4b34-d056-6e36a0b13e1a reconnecting May 18 02:08:17 ALPL202 kernel: Lustre: 7394:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 02:08:26 ALPL202 kernel: LustreError: 7279:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810198653800 x1499587879193166/t0 o8->929aa131-7e43-4b34-d056-6e36a0b13e1a@:0/0 lens 368/264 e 0 to 0 dl 1431886206 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:08:26 ALPL202 kernel: LustreError: 7279:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 02:08:29 ALPL202 kernel: Lustre: 7510:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 18 02:08:29 ALPL202 kernel: Lustre: 7510:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 02:18:17 ALPL202 kernel: Lustre: 7598:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e8917316-444f-9483-db5a-7c7fb1549a1e reconnecting May 18 02:18:17 ALPL202 kernel: Lustre: 7598:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2834 previous similar messages May 18 02:18:26 ALPL202 kernel: LustreError: 6504:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105726b9050 x1499006652841636/t0 o8->e8917316-444f-9483-db5a-7c7fb1549a1e@NET_0x500000a03011e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431886806 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:18:26 ALPL202 kernel: LustreError: 6504:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 18 02:18:29 ALPL202 kernel: Lustre: 7433:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 929aa131-7e43-4b34-d056-6e36a0b13e1a@10.3.1.1@o2ib to 0xffff8105c33c4600; still busy with 3 active RPCs May 18 02:18:29 ALPL202 kernel: Lustre: 7433:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2833 previous similar messages May 18 02:28:18 ALPL202 kernel: Lustre: 6476:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 18 02:28:18 ALPL202 kernel: Lustre: 6476:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 02:28:27 ALPL202 kernel: LustreError: 7420:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105e100f000 x1485800794173200/t0 o8->dee84236-56bc-f2f7-cd2c-ba3d555fd617@NET_0x500000a03011b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431887407 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:28:27 ALPL202 kernel: LustreError: 7420:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 18 02:28:29 ALPL202 kernel: Lustre: 7321:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d3e32527-f91f-d191-fbe6-fbc0e0ba6beb@10.3.1.13@o2ib to 0xffff81021edfda00; still busy with 1 active RPCs May 18 02:28:29 ALPL202 kernel: Lustre: 7321:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 02:38:18 ALPL202 kernel: Lustre: 7319:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f0bae738-4650-c254-ee38-ca124e1eea57 reconnecting May 18 02:38:18 ALPL202 kernel: Lustre: 7319:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2830 previous similar messages May 18 02:38:27 ALPL202 kernel: LustreError: 7587:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81062b819000 x1499649575689874/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431888007 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:38:27 ALPL202 kernel: LustreError: 7587:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2829 previous similar messages May 18 02:38:30 ALPL202 kernel: Lustre: 6438:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from dee84236-56bc-f2f7-cd2c-ba3d555fd617@10.3.1.27@o2ib to 0xffff81050f727a00; still busy with 2 active RPCs May 18 02:38:30 ALPL202 kernel: Lustre: 6438:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2832 previous similar messages May 18 02:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 02:48:18 ALPL202 kernel: Lustre: 7350:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 18 02:48:18 ALPL202 kernel: Lustre: 7350:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 18 02:48:27 ALPL202 kernel: LustreError: 7497:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810343e05800 x1499645797003630/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431888607 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:48:27 ALPL202 kernel: LustreError: 7497:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 18 02:48:30 ALPL202 kernel: Lustre: 7582:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@10.3.1.32@o2ib to 0xffff810c3cf28000; still busy with 3 active RPCs May 18 02:48:30 ALPL202 kernel: Lustre: 7582:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 18 02:58:18 ALPL202 kernel: Lustre: 6454:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04cfad47-91cb-7688-6409-2530587af605 reconnecting May 18 02:58:18 ALPL202 kernel: Lustre: 6454:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 02:58:27 ALPL202 kernel: LustreError: 7659:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81058ab52400 x1499018008734664/t0 o8->04cfad47-91cb-7688-6409-2530587af605@:0/0 lens 368/264 e 0 to 0 dl 1431889207 ref 1 fl Interpret:/0/0 rc -16/0 May 18 02:58:27 ALPL202 kernel: LustreError: 7659:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 18 02:58:30 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d3e32527-f91f-d191-fbe6-fbc0e0ba6beb@10.3.1.13@o2ib to 0xffff81021edfda00; still busy with 1 active RPCs May 18 02:58:30 ALPL202 kernel: Lustre: 7341:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 03:08:18 ALPL202 kernel: Lustre: 7459:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ab7734c9-9219-90ea-5fe9-3f7d86afc6c0 reconnecting May 18 03:08:18 ALPL202 kernel: Lustre: 7459:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2834 previous similar messages May 18 03:08:27 ALPL202 kernel: LustreError: 7657:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105335dbc00 x1499682234221727/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431889807 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:08:27 ALPL202 kernel: LustreError: 7657:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 18 03:08:31 ALPL202 kernel: Lustre: 6443:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@10.3.1.22@o2ib to 0xffff81063fe3a400; still busy with 1 active RPCs May 18 03:08:31 ALPL202 kernel: Lustre: 6443:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 03:18:18 ALPL202 kernel: Lustre: 7380:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4996fcc3-85b7-659a-e4a1-572255254f65 reconnecting May 18 03:18:18 ALPL202 kernel: Lustre: 7380:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 03:18:27 ALPL202 kernel: LustreError: 7652:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810630863c00 x1498183791737340/t0 o8->4996fcc3-85b7-659a-e4a1-572255254f65@NET_0x500000a030119_UUID:0/0 lens 368/264 e 0 to 0 dl 1431890407 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:18:27 ALPL202 kernel: LustreError: 7652:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 18 03:18:31 ALPL202 kernel: Lustre: 7551:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@10.3.1.32@o2ib to 0xffff810c3cf28000; still busy with 3 active RPCs May 18 03:18:31 ALPL202 kernel: Lustre: 7551:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 03:28:18 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 13307ac7-2a42-d2e1-9f17-7cd2822f32c8 reconnecting May 18 03:28:18 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 03:28:27 ALPL202 kernel: LustreError: 7490:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105decb0800 x1497757446307503/t0 o8->13307ac7-2a42-d2e1-9f17-7cd2822f32c8@NET_0x500000a030129_UUID:0/0 lens 368/264 e 0 to 0 dl 1431891007 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:28:27 ALPL202 kernel: LustreError: 7490:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 03:28:31 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from b723915e-bd6d-5de8-3f38-d333eb1db2ed@10.3.1.3@o2ib to 0xffff81036846c200; still busy with 2 active RPCs May 18 03:28:31 ALPL202 kernel: Lustre: 7359:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 03:38:18 ALPL202 kernel: Lustre: 7380:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 1ed5a549-efbb-515c-ff9a-8b8741af8256 reconnecting May 18 03:38:18 ALPL202 kernel: Lustre: 7380:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 03:38:27 ALPL202 kernel: LustreError: 7652:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81062203a800 x1499017965478473/t0 o8->1ed5a549-efbb-515c-ff9a-8b8741af8256@NET_0x500000a030127_UUID:0/0 lens 368/264 e 0 to 0 dl 1431891607 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:38:27 ALPL202 kernel: LustreError: 7652:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 18 03:38:31 ALPL202 kernel: Lustre: 6525:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0@10.3.1.17@o2ib to 0xffff81061f5f1200; still busy with 4 active RPCs May 18 03:38:31 ALPL202 kernel: Lustre: 6525:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 03:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 03:48:19 ALPL202 kernel: Lustre: 7279:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 18 03:48:19 ALPL202 kernel: Lustre: 7279:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 03:48:28 ALPL202 kernel: LustreError: 6435:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103dc042800 x1485800794340899/t0 o8->dee84236-56bc-f2f7-cd2c-ba3d555fd617@NET_0x500000a03011b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431892208 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:48:28 ALPL202 kernel: LustreError: 6435:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 03:48:31 ALPL202 kernel: Lustre: 7509:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04ec28a8-eb95-c793-1a9c-7827d08a2f27@10.3.1.9@o2ib to 0xffff81079778a600; still busy with 4 active RPCs May 18 03:48:31 ALPL202 kernel: Lustre: 7509:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 03:58:19 ALPL202 kernel: Lustre: 7482:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f9ed569f-59c6-0f68-7b91-ef32fbaef2d8 reconnecting May 18 03:58:19 ALPL202 kernel: Lustre: 7482:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 03:58:28 ALPL202 kernel: LustreError: 7564:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106227ff800 x1499597122763167/t0 o8->f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@NET_0x500000a030116_UUID:0/0 lens 368/264 e 0 to 0 dl 1431892808 ref 1 fl Interpret:/0/0 rc -16/0 May 18 03:58:28 ALPL202 kernel: LustreError: 7564:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 03:58:31 ALPL202 kernel: Lustre: 7626:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 4996fcc3-85b7-659a-e4a1-572255254f65@10.3.1.25@o2ib to 0xffff8105c33c4400; still busy with 3 active RPCs May 18 03:58:31 ALPL202 kernel: Lustre: 7626:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 04:08:19 ALPL202 kernel: Lustre: 7572:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: d7a5a345-9841-f534-ddde-a2808cdd637c reconnecting May 18 04:08:19 ALPL202 kernel: Lustre: 7572:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 04:08:28 ALPL202 kernel: LustreError: 7339:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81052e8a8400 x1497851291946527/t0 o8->d7a5a345-9841-f534-ddde-a2808cdd637c@:0/0 lens 368/264 e 0 to 0 dl 1431893408 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:08:28 ALPL202 kernel: LustreError: 7339:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 04:08:31 ALPL202 kernel: Lustre: 7598:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 18 04:08:31 ALPL202 kernel: Lustre: 7598:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 04:18:19 ALPL202 kernel: Lustre: 6450:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f0bae738-4650-c254-ee38-ca124e1eea57 reconnecting May 18 04:18:19 ALPL202 kernel: Lustre: 6450:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 04:18:28 ALPL202 kernel: LustreError: 7535:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104ecb88400 x1499649575925400/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431894008 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:18:28 ALPL202 kernel: LustreError: 7535:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 18 04:18:31 ALPL202 kernel: Lustre: 7321:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ade1e689-13ba-5165-30df-204eabc4dae8@10.3.1.18@o2ib to 0xffff81055247fa00; still busy with 3 active RPCs May 18 04:18:31 ALPL202 kernel: Lustre: 7321:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 04:28:19 ALPL202 kernel: Lustre: 7349:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: efebfe6f-6429-bcd2-fd37-e715eaf8983b reconnecting May 18 04:28:19 ALPL202 kernel: Lustre: 7349:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2835 previous similar messages May 18 04:28:28 ALPL202 kernel: LustreError: 6504:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105f0db7400 x1497853924436502/t0 o8->efebfe6f-6429-bcd2-fd37-e715eaf8983b@NET_0x500000a030124_UUID:0/0 lens 368/264 e 0 to 0 dl 1431894608 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:28:28 ALPL202 kernel: LustreError: 6504:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 18 04:28:31 ALPL202 kernel: Lustre: 7554:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@10.3.1.47@o2ib to 0xffff81021edfdc00; still busy with 1 active RPCs May 18 04:28:31 ALPL202 kernel: Lustre: 7554:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 04:38:19 ALPL202 kernel: Lustre: 7637:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: LFS02-mdtlov_UUID reconnecting May 18 04:38:19 ALPL202 kernel: Lustre: 7637:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2839 previous similar messages May 18 04:38:28 ALPL202 kernel: LustreError: 7456:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106224b5400 x1485800371789658/t0 o8->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431895208 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:38:28 ALPL202 kernel: LustreError: 7456:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 04:38:31 ALPL202 kernel: Lustre: 7472:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e8917316-444f-9483-db5a-7c7fb1549a1e@10.3.1.30@o2ib to 0xffff81021edfd400; still busy with 3 active RPCs May 18 04:38:31 ALPL202 kernel: Lustre: 7472:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2833 previous similar messages May 18 04:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 04:48:20 ALPL202 kernel: Lustre: 7507:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 18 04:48:20 ALPL202 kernel: Lustre: 7507:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 04:48:29 ALPL202 kernel: LustreError: 7620:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81056c9dbc50 x1485800794465946/t0 o8->dee84236-56bc-f2f7-cd2c-ba3d555fd617@NET_0x500000a03011b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431895809 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:48:29 ALPL202 kernel: LustreError: 7620:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 18 04:48:31 ALPL202 kernel: Lustre: 7548:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f32cf79c-e087-bfe3-6e8f-7fde95ab168b@10.3.1.15@o2ib to 0xffff8105c33c4e00; still busy with 3 active RPCs May 18 04:48:31 ALPL202 kernel: Lustre: 7548:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 04:58:20 ALPL202 kernel: Lustre: 7621:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ab7734c9-9219-90ea-5fe9-3f7d86afc6c0 reconnecting May 18 04:58:20 ALPL202 kernel: Lustre: 7621:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 04:58:29 ALPL202 kernel: LustreError: 7425:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810321260400 x1499682234476816/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431896409 ref 1 fl Interpret:/0/0 rc -16/0 May 18 04:58:29 ALPL202 kernel: LustreError: 7425:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2831 previous similar messages May 18 04:58:32 ALPL202 kernel: Lustre: 7330:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 04cfad47-91cb-7688-6409-2530587af605@10.3.1.48@o2ib to 0xffff810479a1a600; still busy with 2 active RPCs May 18 04:58:32 ALPL202 kernel: Lustre: 7330:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 05:08:20 ALPL202 kernel: Lustre: 7490:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: b723915e-bd6d-5de8-3f38-d333eb1db2ed reconnecting May 18 05:08:20 ALPL202 kernel: Lustre: 7490:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2840 previous similar messages May 18 05:08:29 ALPL202 kernel: LustreError: 7639:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104acb83000 x1499653317605697/t0 o8->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 368/264 e 0 to 0 dl 1431897009 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:08:29 ALPL202 kernel: LustreError: 7639:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2839 previous similar messages May 18 05:08:32 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 13307ac7-2a42-d2e1-9f17-7cd2822f32c8@10.3.1.41@o2ib to 0xffff81063f875e00; still busy with 4 active RPCs May 18 05:08:32 ALPL202 kernel: Lustre: 7612:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 18 05:18:20 ALPL202 kernel: Lustre: 6446:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ab7734c9-9219-90ea-5fe9-3f7d86afc6c0 reconnecting May 18 05:18:20 ALPL202 kernel: Lustre: 6446:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 05:18:29 ALPL202 kernel: LustreError: 6441:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810622b28400 x1499682234522959/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431897609 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:18:29 ALPL202 kernel: LustreError: 6441:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 18 05:18:32 ALPL202 kernel: Lustre: 7416:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 72657e2e-01a5-e000-c945-6edb35b026e6@10.3.1.42@o2ib to 0xffff8103c7e72a00; still busy with 1 active RPCs May 18 05:18:32 ALPL202 kernel: Lustre: 7416:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2840 previous similar messages May 18 05:28:20 ALPL202 kernel: Lustre: 7474:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 18 05:28:20 ALPL202 kernel: Lustre: 7474:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 05:28:29 ALPL202 kernel: LustreError: 7459:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff81063002ac00 x1486456504235665/t0 o8->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 368/264 e 0 to 0 dl 1431898209 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:28:29 ALPL202 kernel: LustreError: 7459:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 05:28:32 ALPL202 kernel: Lustre: 7361:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e986becc-307e-f48d-480a-444f9c6b233c@10.3.1.35@o2ib to 0xffff81017d1a0600; still busy with 4 active RPCs May 18 05:28:32 ALPL202 kernel: Lustre: 7361:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2831 previous similar messages May 18 05:38:21 ALPL202 kernel: Lustre: 7626:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: dee84236-56bc-f2f7-cd2c-ba3d555fd617 reconnecting May 18 05:38:21 ALPL202 kernel: Lustre: 7626:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2842 previous similar messages May 18 05:38:29 ALPL202 kernel: LustreError: 7616:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104aec94c00 x1499775199150913/t0 o8->f20e7e2b-9669-0314-8db8-3d5538bfde76@:0/0 lens 368/264 e 0 to 0 dl 1431898809 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:38:29 ALPL202 kernel: LustreError: 7616:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2841 previous similar messages May 18 05:38:32 ALPL202 kernel: Lustre: 6402:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from fc0811da-8123-3956-a1b8-e1d99a941c14@10.3.1.28@o2ib to 0xffff8105d807f000; still busy with 4 active RPCs May 18 05:38:32 ALPL202 kernel: Lustre: 6402:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2836 previous similar messages May 18 05:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 05:48:21 ALPL202 kernel: Lustre: 6410:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 22492cf0-9baa-5c7d-4f94-ca5f8ab20776 reconnecting May 18 05:48:21 ALPL202 kernel: Lustre: 6410:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2830 previous similar messages May 18 05:48:29 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101d3804c00 x1499587879729309/t0 o8->929aa131-7e43-4b34-d056-6e36a0b13e1a@:0/0 lens 368/264 e 0 to 0 dl 1431899409 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:48:29 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2828 previous similar messages May 18 05:48:32 ALPL202 kernel: Lustre: 7603:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 18 05:48:32 ALPL202 kernel: Lustre: 7603:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 05:58:21 ALPL202 kernel: Lustre: 7307:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 18 05:58:21 ALPL202 kernel: Lustre: 7307:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2840 previous similar messages May 18 05:58:29 ALPL202 kernel: LustreError: 7492:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105043c3400 x1499004085950026/t0 o8->e3633471-5b20-ed6f-3476-b9c3d5d78060@NET_0x500000a030105_UUID:0/0 lens 368/264 e 0 to 0 dl 1431900009 ref 1 fl Interpret:/0/0 rc -16/0 May 18 05:58:29 ALPL202 kernel: LustreError: 7492:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2841 previous similar messages May 18 05:58:33 ALPL202 kernel: Lustre: 7589:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from d7a5a345-9841-f534-ddde-a2808cdd637c@10.3.1.34@o2ib to 0xffff81061f5f1600; still busy with 3 active RPCs May 18 05:58:33 ALPL202 kernel: Lustre: 7589:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2829 previous similar messages May 18 06:08:21 ALPL202 kernel: Lustre: 7360:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 18 06:08:21 ALPL202 kernel: Lustre: 7360:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 06:08:29 ALPL202 kernel: LustreError: 7595:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103c80b9000 x1499014178481991/t0 o8->9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d@NET_0x500000a03012f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431900609 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:08:29 ALPL202 kernel: LustreError: 7595:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 18 06:08:33 ALPL202 kernel: Lustre: 6442:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ade1e689-13ba-5165-30df-204eabc4dae8@10.3.1.18@o2ib to 0xffff81055247fa00; still busy with 3 active RPCs May 18 06:08:33 ALPL202 kernel: Lustre: 6442:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2845 previous similar messages May 18 06:18:21 ALPL202 kernel: Lustre: 6505:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 929aa131-7e43-4b34-d056-6e36a0b13e1a reconnecting May 18 06:18:21 ALPL202 kernel: Lustre: 6505:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2829 previous similar messages May 18 06:18:30 ALPL202 kernel: LustreError: 7401:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8100a854c000 x1497851292252472/t0 o8->d7a5a345-9841-f534-ddde-a2808cdd637c@:0/0 lens 368/264 e 0 to 0 dl 1431901210 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:18:30 ALPL202 kernel: LustreError: 7401:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2834 previous similar messages May 18 06:18:33 ALPL202 kernel: Lustre: 7454:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 17342f99-c92f-d6af-0247-fdddc176fe9b@10.3.1.2@o2ib to 0xffff81061f5f1800; still busy with 1 active RPCs May 18 06:18:33 ALPL202 kernel: Lustre: 7454:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 06:28:21 ALPL202 kernel: Lustre: 7474:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e3633471-5b20-ed6f-3476-b9c3d5d78060 reconnecting May 18 06:28:21 ALPL202 kernel: Lustre: 7474:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2839 previous similar messages May 18 06:28:30 ALPL202 kernel: LustreError: 7590:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105d1b68000 x1499653317798161/t0 o8->b723915e-bd6d-5de8-3f38-d333eb1db2ed@NET_0x500000a030103_UUID:0/0 lens 368/264 e 0 to 0 dl 1431901810 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:28:30 ALPL202 kernel: LustreError: 7590:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 06:28:34 ALPL202 kernel: Lustre: 7430:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ea79a342-578d-a338-caef-c51d0fe5e904@10.3.1.46@o2ib to 0xffff810372190200; still busy with 2 active RPCs May 18 06:28:34 ALPL202 kernel: Lustre: 7430:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2830 previous similar messages May 18 06:38:21 ALPL202 kernel: Lustre: 7281:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 9466f5e8-8ddc-4d7a-b25d-61d9bb7aed8d reconnecting May 18 06:38:21 ALPL202 kernel: Lustre: 7281:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 06:38:30 ALPL202 kernel: LustreError: 6464:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105495d1c00 x1499645797479471/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431902410 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:38:30 ALPL202 kernel: LustreError: 6464:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 18 06:38:34 ALPL202 kernel: Lustre: 7413:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 18 06:38:34 ALPL202 kernel: Lustre: 7413:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2842 previous similar messages May 18 06:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 06:48:22 ALPL202 kernel: Lustre: 7615:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: fc0811da-8123-3956-a1b8-e1d99a941c14 reconnecting May 18 06:48:22 ALPL202 kernel: Lustre: 7615:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2829 previous similar messages May 18 06:48:30 ALPL202 kernel: LustreError: 7441:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104a1420850 x1486456504391570/t0 o8->04ec28a8-eb95-c793-1a9c-7827d08a2f27@:0/0 lens 368/264 e 0 to 0 dl 1431903010 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:48:30 ALPL202 kernel: LustreError: 7441:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2835 previous similar messages May 18 06:48:34 ALPL202 kernel: Lustre: 7600:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e8917316-444f-9483-db5a-7c7fb1549a1e@10.3.1.30@o2ib to 0xffff81021edfd400; still busy with 3 active RPCs May 18 06:48:34 ALPL202 kernel: Lustre: 7600:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 06:58:22 ALPL202 kernel: Lustre: 6459:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f0bae738-4650-c254-ee38-ca124e1eea57 reconnecting May 18 06:58:22 ALPL202 kernel: Lustre: 6459:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2841 previous similar messages May 18 06:58:30 ALPL202 kernel: LustreError: 7341:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105c91bf800 x1499566482184306/t0 o8->f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@NET_0x500000a03012b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431903610 ref 1 fl Interpret:/0/0 rc -16/0 May 18 06:58:30 ALPL202 kernel: LustreError: 7341:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 06:58:35 ALPL202 kernel: Lustre: 7628:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84@10.3.1.24@o2ib to 0xffff810ab8f72c00; still busy with 1 active RPCs May 18 06:58:35 ALPL202 kernel: Lustre: 7628:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 07:08:22 ALPL202 kernel: Lustre: 7481:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 929aa131-7e43-4b34-d056-6e36a0b13e1a reconnecting May 18 07:08:22 ALPL202 kernel: Lustre: 7481:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 18 07:08:30 ALPL202 kernel: LustreError: 7528:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810055f76000 x1487072295678639/t0 o8->86d0ab2a-246f-a784-2744-c50f68cee379@NET_0x500000a03011a_UUID:0/0 lens 368/264 e 0 to 0 dl 1431904210 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:08:30 ALPL202 kernel: LustreError: 7528:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2831 previous similar messages May 18 07:08:35 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from LFS02-mdtlov_UUID@10.3.2.72@o2ib to 0xffff810517df7a00; still busy with 1 active RPCs May 18 07:08:35 ALPL202 kernel: Lustre: 7371:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2838 previous similar messages May 18 07:18:22 ALPL202 kernel: Lustre: 7420:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: e3633471-5b20-ed6f-3476-b9c3d5d78060 reconnecting May 18 07:18:22 ALPL202 kernel: Lustre: 7420:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 07:18:30 ALPL202 kernel: LustreError: 7328:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103233a8000 x1499839928028094/t0 o8->LFS02-mdtlov_UUID@NET_0x500000a030248_UUID:0/0 lens 368/264 e 0 to 0 dl 1431904810 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:18:30 ALPL202 kernel: LustreError: 7328:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 18 07:18:35 ALPL202 kernel: Lustre: 7407:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e8917316-444f-9483-db5a-7c7fb1549a1e@10.3.1.30@o2ib to 0xffff81021edfd400; still busy with 3 active RPCs May 18 07:18:35 ALPL202 kernel: Lustre: 7407:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 07:28:23 ALPL202 kernel: Lustre: 6431:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 18 07:28:23 ALPL202 kernel: Lustre: 6431:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2840 previous similar messages May 18 07:28:30 ALPL202 kernel: LustreError: 6465:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8101aa413850 x1499017965978578/t0 o8->1ed5a549-efbb-515c-ff9a-8b8741af8256@NET_0x500000a030127_UUID:0/0 lens 368/264 e 0 to 0 dl 1431905410 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:28:30 ALPL202 kernel: LustreError: 6465:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 18 07:28:35 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from e3633471-5b20-ed6f-3476-b9c3d5d78060@10.3.1.5@o2ib to 0xffff81054b3a3400; still busy with 3 active RPCs May 18 07:28:35 ALPL202 kernel: Lustre: 6469:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 07:38:23 ALPL202 kernel: Lustre: 6430:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: ab7734c9-9219-90ea-5fe9-3f7d86afc6c0 reconnecting May 18 07:38:23 ALPL202 kernel: Lustre: 6430:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 18 07:38:31 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103d41c7800 x1499597159824758/t0 o8->f9ed569f-59c6-0f68-7b91-ef32fbaef2d8@NET_0x500000a030116_UUID:0/0 lens 368/264 e 0 to 0 dl 1431906011 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:38:31 ALPL202 kernel: LustreError: 6467:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2831 previous similar messages May 18 07:38:35 ALPL202 kernel: Lustre: 7532:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@10.3.1.43@o2ib to 0xffff81063fe3aa00; still busy with 4 active RPCs May 18 07:38:35 ALPL202 kernel: Lustre: 7532:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2841 previous similar messages May 18 07:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 07:48:24 ALPL202 kernel: Lustre: 7506:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: a1950f92-7e5d-7d7b-af8e-ae39c24d40ed reconnecting May 18 07:48:24 ALPL202 kernel: Lustre: 7506:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 07:48:31 ALPL202 kernel: LustreError: 7294:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810631928800 x1485800372132653/t0 o8->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431906611 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:48:31 ALPL202 kernel: LustreError: 7294:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2842 previous similar messages May 18 07:48:35 ALPL202 kernel: Lustre: 7457:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 02af742f-5725-28cc-1be3-577b0076f6d3@10.3.1.44@o2ib to 0xffff81050f727e00; still busy with 1 active RPCs May 18 07:48:35 ALPL202 kernel: Lustre: 7457:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2830 previous similar messages May 18 07:58:24 ALPL202 kernel: Lustre: 7408:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: de0fa4c5-b63e-b22c-6d6a-d2383f854657 reconnecting May 18 07:58:24 ALPL202 kernel: Lustre: 7408:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2844 previous similar messages May 18 07:58:32 ALPL202 kernel: LustreError: 7482:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105498b6c50 x1499649576443256/t0 o8->f0bae738-4650-c254-ee38-ca124e1eea57@:0/0 lens 368/264 e 0 to 0 dl 1431907212 ref 1 fl Interpret:/0/0 rc -16/0 May 18 07:58:32 ALPL202 kernel: LustreError: 7482:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2839 previous similar messages May 18 07:58:36 ALPL202 kernel: Lustre: 7574:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from a1950f92-7e5d-7d7b-af8e-ae39c24d40ed@10.3.1.21@o2ib to 0xffff810165d97e00; still busy with 2 active RPCs May 18 07:58:36 ALPL202 kernel: Lustre: 7574:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 08:08:24 ALPL202 kernel: Lustre: 7563:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: f20e7e2b-9669-0314-8db8-3d5538bfde76 reconnecting May 18 08:08:24 ALPL202 kernel: Lustre: 7563:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2829 previous similar messages May 18 08:08:32 ALPL202 kernel: LustreError: 7646:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8106226a8c00 x1497757446972397/t0 o8->13307ac7-2a42-d2e1-9f17-7cd2822f32c8@NET_0x500000a030129_UUID:0/0 lens 368/264 e 0 to 0 dl 1431907812 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:08:32 ALPL202 kernel: LustreError: 7646:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2830 previous similar messages May 18 08:08:36 ALPL202 kernel: Lustre: 7494:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 47872a6f-a0e3-1e39-23f4-a2f8dbea0e84@10.3.1.24@o2ib to 0xffff810ab8f72c00; still busy with 1 active RPCs May 18 08:08:36 ALPL202 kernel: Lustre: 7494:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2844 previous similar messages May 18 08:18:24 ALPL202 kernel: Lustre: 7318:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 1ed5a549-efbb-515c-ff9a-8b8741af8256 reconnecting May 18 08:18:24 ALPL202 kernel: Lustre: 7318:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 08:18:32 ALPL202 kernel: LustreError: 7471:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103fde5e800 x1499566482372987/t0 o8->f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@NET_0x500000a03012b_UUID:0/0 lens 368/264 e 0 to 0 dl 1431908412 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:18:32 ALPL202 kernel: LustreError: 7471:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2840 previous similar messages May 18 08:18:37 ALPL202 kernel: Lustre: 7375:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 22492cf0-9baa-5c7d-4f94-ca5f8ab20776@10.3.1.37@o2ib to 0xffff810372190800; still busy with 4 active RPCs May 18 08:18:37 ALPL202 kernel: Lustre: 7375:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2829 previous similar messages May 18 08:28:25 ALPL202 kernel: Lustre: 7348:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 127a567c-2e83-70ca-5c5a-99111e1a5592 reconnecting May 18 08:28:25 ALPL202 kernel: Lustre: 7348:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 08:28:32 ALPL202 kernel: LustreError: 6407:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8104741f6c00 x1499006653745076/t0 o8->e8917316-444f-9483-db5a-7c7fb1549a1e@NET_0x500000a03011e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431909012 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:28:32 ALPL202 kernel: LustreError: 6407:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 08:28:37 ALPL202 kernel: Lustre: 7575:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ade1e689-13ba-5165-30df-204eabc4dae8@10.3.1.18@o2ib to 0xffff81055247fa00; still busy with 3 active RPCs May 18 08:28:37 ALPL202 kernel: Lustre: 7575:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2842 previous similar messages May 18 08:38:25 ALPL202 kernel: Lustre: 7561:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: b723915e-bd6d-5de8-3f38-d333eb1db2ed reconnecting May 18 08:38:25 ALPL202 kernel: Lustre: 7561:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2833 previous similar messages May 18 08:38:32 ALPL202 kernel: LustreError: 6403:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103b3ca6c00 x1485800372223155/t0 o8->f32cf79c-e087-bfe3-6e8f-7fde95ab168b@NET_0x500000a03010f_UUID:0/0 lens 368/264 e 0 to 0 dl 1431909612 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:38:32 ALPL202 kernel: LustreError: 6403:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2832 previous similar messages May 18 08:38:37 ALPL202 kernel: Lustre: 6412:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f20e7e2b-9669-0314-8db8-3d5538bfde76@10.3.1.66@o2ib to 0xffff8105e5c98200; still busy with 8 active RPCs May 18 08:38:37 ALPL202 kernel: Lustre: 6412:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2835 previous similar messages May 18 08:44:46 ALPL202 : error getting update info: Cannot find a valid baseurl for repo: base May 18 08:48:25 ALPL202 kernel: Lustre: 7584:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: efebfe6f-6429-bcd2-fd37-e715eaf8983b reconnecting May 18 08:48:25 ALPL202 kernel: Lustre: 7584:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2834 previous similar messages May 18 08:48:33 ALPL202 kernel: LustreError: 7326:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810621c07800 x1499645797747245/t0 o8->127a567c-2e83-70ca-5c5a-99111e1a5592@NET_0x500000a03010e_UUID:0/0 lens 368/264 e 0 to 0 dl 1431910213 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:48:33 ALPL202 kernel: LustreError: 7326:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2837 previous similar messages May 18 08:48:37 ALPL202 kernel: Lustre: 6489:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from 929aa131-7e43-4b34-d056-6e36a0b13e1a@10.3.1.1@o2ib to 0xffff8105c33c4600; still busy with 3 active RPCs May 18 08:48:37 ALPL202 kernel: Lustre: 6489:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2833 previous similar messages May 18 08:58:25 ALPL202 kernel: Lustre: 6477:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 4f5de4fd-e7f6-d5f3-73a6-9864332b45d0 reconnecting May 18 08:58:25 ALPL202 kernel: Lustre: 6477:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2838 previous similar messages May 18 08:58:33 ALPL202 kernel: LustreError: 7368:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8103a3bfd000 x1499682235032906/t0 o8->ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@NET_0x500000a030120_UUID:0/0 lens 368/264 e 0 to 0 dl 1431910813 ref 1 fl Interpret:/0/0 rc -16/0 May 18 08:58:33 ALPL202 kernel: LustreError: 7368:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2833 previous similar messages May 18 08:58:37 ALPL202 kernel: Lustre: 7441:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from f9fdf5c1-f5d4-d701-81d8-f6afd06169ef@10.3.1.43@o2ib to 0xffff81063fe3aa00; still busy with 4 active RPCs May 18 08:58:37 ALPL202 kernel: Lustre: 7441:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 09:08:25 ALPL202 kernel: Lustre: 7513:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 04ec28a8-eb95-c793-1a9c-7827d08a2f27 reconnecting May 18 09:08:25 ALPL202 kernel: Lustre: 7513:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2831 previous similar messages May 18 09:08:33 ALPL202 kernel: LustreError: 7521:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff8105dceb0800 x1499033022573656/t0 o8->79a59854-31b9-d753-4e22-8912905a5085@:0/0 lens 368/264 e 0 to 0 dl 1431911413 ref 1 fl Interpret:/0/0 rc -16/0 May 18 09:08:33 ALPL202 kernel: LustreError: 7521:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2838 previous similar messages May 18 09:08:38 ALPL202 kernel: Lustre: 7423:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab1586ab-f630-d76b-7485-396b47d527b1@10.3.1.7@o2ib to 0xffff8105db9f0600; still busy with 3 active RPCs May 18 09:08:38 ALPL202 kernel: Lustre: 7423:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2837 previous similar messages May 18 09:18:25 ALPL202 kernel: Lustre: 6506:0:(ldlm_lib.c:576:target_handle_reconnect()) LFS02-OST0005: 2a3fd8ac-ad9e-6c5c-eb5a-e9e64491e0ac reconnecting May 18 09:18:25 ALPL202 kernel: Lustre: 6506:0:(ldlm_lib.c:576:target_handle_reconnect()) Skipped 2836 previous similar messages May 18 09:18:33 ALPL202 kernel: LustreError: 7658:0:(ldlm_lib.c:1921:target_send_reply_msg()) @@@ processing error (-16) req@ffff810198653c00 x1499010344090436/t0 o8->02af742f-5725-28cc-1be3-577b0076f6d3@NET_0x500000a03012c_UUID:0/0 lens 368/264 e 0 to 0 dl 1431912013 ref 1 fl Interpret:/0/0 rc -16/0 May 18 09:18:33 ALPL202 kernel: LustreError: 7658:0:(ldlm_lib.c:1921:target_send_reply_msg()) Skipped 2836 previous similar messages May 18 09:18:39 ALPL202 kernel: Lustre: 7600:0:(ldlm_lib.c:876:target_handle_connect()) LFS02-OST0005: refuse reconnection from ab7734c9-9219-90ea-5fe9-3f7d86afc6c0@10.3.1.32@o2ib to 0xffff810c3cf28000; still busy with 3 active RPCs May 18 09:18:39 ALPL202 kernel: Lustre: 7600:0:(ldlm_lib.c:876:target_handle_connect()) Skipped 2834 previous similar messages May 18 09:23:05 ALPL202 syslogd 1.4.1: restart. May 18 09:23:05 ALPL202 kernel: klogd 1.4.1, log source = /proc/kmsg started. May 18 09:23:05 ALPL202 kernel: Linux version 2.6.18-348.3.1.el5_lustre.1.8.9.ddn8 (root@build07) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)) #1 SMP Thu Feb 12 14:53:40 JST 2015 May 18 09:23:05 ALPL202 kernel: Command line: ro root=/dev/md1 rhgb quiet May 18 09:23:05 ALPL202 kernel: BIOS-provided physical RAM map: May 18 09:23:05 ALPL202 kernel: BIOS-e820: 0000000000010000 - 0000000000098400 (usable) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 0000000000098400 - 00000000000a0000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 0000000000100000 - 00000000bf780000 (usable) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000bf78e000 - 00000000bf790000 type 9 May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000bf790000 - 00000000bf79e000 (ACPI data) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000bf79e000 - 00000000bf7d0000 (ACPI NVS) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000bf7d0000 - 00000000bf7e0000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000bf7ec000 - 00000000c0000000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000fed20000 - 00000000fed40000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved) May 18 09:23:05 ALPL202 kernel: BIOS-e820: 0000000100000000 - 0000000c40000000 (usable) May 18 09:23:05 ALPL202 kernel: DMI present. May 18 09:23:05 ALPL202 kernel: SRAT: PXM 0 -> APIC 0 -> Node 0 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 0 -> APIC 2 -> Node 0 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 0 -> APIC 18 -> Node 0 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 0 -> APIC 20 -> Node 0 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 1 -> APIC 32 -> Node 1 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 1 -> APIC 34 -> Node 1 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 1 -> APIC 50 -> Node 1 May 18 09:23:05 ALPL202 kernel: SRAT: PXM 1 -> APIC 52 -> Node 1 May 18 09:23:05 ALPL202 kernel: SRAT: Node 0 PXM 0 0-a0000 May 18 09:23:05 ALPL202 kernel: SRAT: Node 0 PXM 0 0-c0000000 May 18 09:23:05 ALPL202 kernel: SRAT: Node 0 PXM 0 0-640000000 May 18 09:23:05 ALPL202 kernel: SRAT: Node 1 PXM 1 640000000-c40000000 May 18 09:23:05 ALPL202 kernel: Bootmem setup node 0 0000000000000000-0000000640000000 May 18 09:23:05 ALPL202 kernel: Bootmem setup node 1 0000000640000000-0000000c40000000 May 18 09:23:05 ALPL202 kernel: Memory for crash kernel (0x0 to 0x0) notwithin permissible range May 18 09:23:05 ALPL202 kernel: disabling kdump May 18 09:23:06 ALPL202 kernel: ACPI: PM-Timer IO Port: 0x808 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) May 18 09:23:06 ALPL202 kernel: Processor #0 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) May 18 09:23:06 ALPL202 kernel: Processor #2 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x03] lapic_id[0x12] enabled) May 18 09:23:06 ALPL202 kernel: Processor #18 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x04] lapic_id[0x14] enabled) May 18 09:23:06 ALPL202 kernel: Processor #20 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x05] lapic_id[0x20] enabled) May 18 09:23:06 ALPL202 kernel: Processor #32 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x06] lapic_id[0x22] enabled) May 18 09:23:06 ALPL202 kernel: Processor #34 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x07] lapic_id[0x32] enabled) May 18 09:23:06 ALPL202 kernel: Processor #50 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x08] lapic_id[0x34] enabled) May 18 09:23:06 ALPL202 kernel: Processor #52 6:12 APIC version 21 May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x09] lapic_id[0x88] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x89] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x8a] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x8b] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x8c] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x8d] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x8e] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x10] lapic_id[0x8f] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x11] lapic_id[0x90] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x12] lapic_id[0x91] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x13] lapic_id[0x92] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x14] lapic_id[0x93] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x15] lapic_id[0x94] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x16] lapic_id[0x95] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x17] lapic_id[0x96] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC (acpi_id[0x18] lapic_id[0x97] disabled) May 18 09:23:06 ALPL202 kernel: ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1]) May 18 09:23:06 ALPL202 kernel: ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0]) May 18 09:23:06 ALPL202 kernel: IOAPIC[0]: apic_id 1, version 32, address 0xfec00000, GSI 0-23 May 18 09:23:06 ALPL202 kernel: ACPI: IOAPIC (id[0x03] address[0xfec8a000] gsi_base[24]) May 18 09:23:06 ALPL202 kernel: IOAPIC[1]: apic_id 3, version 32, address 0xfec8a000, GSI 24-47 May 18 09:23:06 ALPL202 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) May 18 09:23:06 ALPL202 kernel: ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) May 18 09:23:06 ALPL202 kernel: Setting APIC routing to physical flat May 18 09:23:06 ALPL202 kernel: ACPI: HPET id: 0x8086a301 base: 0xfed00000 May 18 09:23:06 ALPL202 kernel: Using ACPI (MADT) for SMP configuration information May 18 09:23:06 ALPL202 kernel: Nosave address range: 0000000000098000 - 0000000000099000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 0000000000099000 - 00000000000a0000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000000a0000 - 00000000000e0000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000000e0000 - 0000000000100000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf780000 - 00000000bf78e000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf78e000 - 00000000bf790000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf790000 - 00000000bf79e000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf79e000 - 00000000bf7d0000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf7d0000 - 00000000bf7e0000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf7e0000 - 00000000bf7ec000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000bf7ec000 - 00000000c0000000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000c0000000 - 00000000e0000000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000e0000000 - 00000000f0000000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000f0000000 - 00000000fed20000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000fed20000 - 00000000fed40000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000fed40000 - 00000000fee00000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000fee00000 - 00000000fee01000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000fee01000 - 00000000ffc00000 May 18 09:23:06 ALPL202 kernel: Nosave address range: 00000000ffc00000 - 0000000100000000 May 18 09:23:06 ALPL202 kernel: Allocating PCI resources starting at c2000000 (gap: c0000000:20000000) May 18 09:23:06 ALPL202 kernel: SMP: Allowing 24 CPUs, 16 hotplug CPUs May 18 09:23:06 ALPL202 kernel: Built 2 zonelists. Total pages: 12403446 May 18 09:23:06 ALPL202 kernel: Kernel command line: ro root=/dev/md1 rhgb quiet May 18 09:23:06 ALPL202 kernel: Initializing CPU#0 May 18 09:23:06 ALPL202 kernel: PID hash table entries: 4096 (order: 12, 32768 bytes) May 18 09:23:06 ALPL202 kernel: Console: colour VGA+ 80x25 May 18 09:23:06 ALPL202 kernel: Dentry cache hash table entries: 8388608 (order: 14, 67108864 bytes) May 18 09:23:06 ALPL202 kernel: Inode-cache hash table entries: 4194304 (order: 13, 33554432 bytes) May 18 09:23:06 ALPL202 kernel: Checking aperture... May 18 09:23:06 ALPL202 kernel: ACPI: DMAR not present May 18 09:23:06 ALPL202 kernel: PCI-DMA: Using software bounce buffering for IO (SWIOTLB) May 18 09:23:06 ALPL202 kernel: Placing software IO TLB between 0x707f000 - 0xb07f000 May 18 09:23:06 ALPL202 kernel: Memory: 49445488k/51380224k available (2629k kernel code, 876976k reserved, 1680k data, 224k init) May 18 09:23:06 ALPL202 kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 4800.29 BogoMIPS (lpj=2400148) May 18 09:23:06 ALPL202 kernel: Security Framework v1.0.0 initialized May 18 09:23:06 ALPL202 kernel: SELinux: Initializing. May 18 09:23:06 ALPL202 kernel: selinux_register_security: Registering secondary module capability May 18 09:23:06 ALPL202 kernel: Capability LSM initialized as secondary May 18 09:23:06 ALPL202 kernel: Mount-cache hash table entries: 256 May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 0/0 -> Node 0 May 18 09:23:06 ALPL202 kernel: using mwait in idle threads. May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 0 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 0 May 18 09:23:06 ALPL202 kernel: MCE: Machine Check Exception Reporting is disabled. May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to UP code May 18 09:23:06 ALPL202 kernel: ACPI: Core revision 20060707 May 18 09:23:06 ALPL202 kernel: Using local APIC timer interrupts. May 18 09:23:06 ALPL202 kernel: Detected 8.333 MHz APIC timer. May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 1/8 APIC 0x2 May 18 09:23:06 ALPL202 kernel: Initializing CPU#1 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.07 BogoMIPS (lpj=2400036) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 1/2 -> Node 0 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 0 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 1 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 2/8 APIC 0x12 May 18 09:23:06 ALPL202 kernel: Initializing CPU#2 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.14 BogoMIPS (lpj=2400071) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 2/12 -> Node 0 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 0 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 9 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 3/8 APIC 0x14 May 18 09:23:06 ALPL202 kernel: Initializing CPU#3 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.28 BogoMIPS (lpj=2400140) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 3/14 -> Node 0 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 0 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 10 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 4/8 APIC 0x20 May 18 09:23:06 ALPL202 kernel: Initializing CPU#4 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.11 BogoMIPS (lpj=2400056) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 4/20 -> Node 1 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 1 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 0 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 5/8 APIC 0x22 May 18 09:23:06 ALPL202 kernel: Initializing CPU#5 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.11 BogoMIPS (lpj=2400055) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 5/22 -> Node 1 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 1 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 1 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 6/8 APIC 0x32 May 18 09:23:06 ALPL202 kernel: Initializing CPU#6 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.32 BogoMIPS (lpj=2400160) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 6/32 -> Node 1 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 1 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 9 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: SMP alternatives: switching to SMP code May 18 09:23:06 ALPL202 kernel: Booting processor 7/8 APIC 0x34 May 18 09:23:06 ALPL202 kernel: Initializing CPU#7 May 18 09:23:06 ALPL202 kernel: Calibrating delay using timer specific routine.. 4800.25 BogoMIPS (lpj=2400125) May 18 09:23:06 ALPL202 kernel: CPU: L1 I cache: 32K, L1 D cache: 32K May 18 09:23:06 ALPL202 kernel: CPU: L2 cache: 256K May 18 09:23:06 ALPL202 kernel: CPU: L3 cache: 12288K May 18 09:23:06 ALPL202 kernel: CPU 7/34 -> Node 1 May 18 09:23:06 ALPL202 kernel: CPU: Physical Processor ID: 1 May 18 09:23:06 ALPL202 kernel: CPU: Processor Core ID: 10 May 18 09:23:06 ALPL202 kernel: Intel(R) Xeon(R) CPU E5620 @ 2.40GHz stepping 02 May 18 09:23:06 ALPL202 kernel: Brought up 8 CPUs May 18 09:23:06 ALPL202 kernel: NMI watchdog testing PASSED. May 18 09:23:06 ALPL202 kernel: time.c: Using 14.318180 MHz WALL HPET GTOD HPET/TSC timer. May 18 09:23:06 ALPL202 kernel: time.c: Detected 2400.148 MHz processor. May 18 09:23:06 ALPL202 kernel: migration_cost=20,1515 May 18 09:23:06 ALPL202 kernel: checking if image is initramfs... it is May 18 09:23:06 ALPL202 kernel: Freeing initrd memory: 2697k freed May 18 09:23:06 ALPL202 kernel: NET: Registered protocol family 16 May 18 09:23:06 ALPL202 kernel: ACPI: bus type pci registered May 18 09:23:06 ALPL202 kernel: PCI: Using MMCONFIG at e0000000 May 18 09:23:06 ALPL202 kernel: ACPI: Interpreter enabled May 18 09:23:06 ALPL202 kernel: ACPI: Using IOAPIC for interrupt routing May 18 09:23:06 ALPL202 kernel: ACPI: No dock devices found. May 18 09:23:06 ALPL202 kernel: ACPI: PCI Root Bridge [PCI0] (0000:00) May 18 09:23:06 ALPL202 kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01] May 18 09:23:06 ALPL202 kernel: pci 0000:00:03.0: PCI bridge to [bus 02-02] May 18 09:23:06 ALPL202 kernel: pci 0000:00:05.0: PCI bridge to [bus 03-03] May 18 09:23:06 ALPL202 kernel: pci 0000:00:07.0: PCI bridge to [bus 04-04] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.0: PCI bridge to [bus 07-08] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.4: PCI bridge to [bus 05-06] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1e.0: PCI bridge to [bus 09-09] (subtractive decode) May 18 09:23:06 ALPL202 kernel: bus 0 -> pxm 0 -> node 0 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 6 7 *10 11 12 14 15) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 6 7 10 11 12 14 *15) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 6 7 10 *11 12 14 15) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 6 7 10 11 12 *14 15) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled. May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 6 *7 10 11 12 14 15) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 6 7 10 11 12 14 15) *0, disabled. May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *6 7 10 11 12 14 15) May 18 09:23:06 ALPL202 kernel: Linux Plug and Play Support v0.97 (c) Adam Belay May 18 09:23:06 ALPL202 kernel: pnp: PnP ACPI init May 18 09:23:06 ALPL202 kernel: pnp: PnP ACPI: found 14 devices May 18 09:23:06 ALPL202 kernel: usbcore: registered new driver usbfs May 18 09:23:06 ALPL202 kernel: usbcore: registered new driver hub May 18 09:23:06 ALPL202 kernel: PCI: Using ACPI for IRQ routing May 18 09:23:06 ALPL202 kernel: PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report May 18 09:23:06 ALPL202 kernel: NetLabel: Initializing May 18 09:23:06 ALPL202 kernel: NetLabel: domain hash size = 128 May 18 09:23:06 ALPL202 kernel: NetLabel: protocols = UNLABELED CIPSOv4 May 18 09:23:06 ALPL202 kernel: NetLabel: unlabeled traffic allowed by default May 18 09:23:06 ALPL202 kernel: hpet0: at MMIO 0xfed00000 (virtual 0xffffffffff5fe000), IRQs 2, 8, 0, 0 May 18 09:23:06 ALPL202 kernel: hpet0: 4 64-bit timers, 14318180 Hz May 18 09:23:06 ALPL202 kernel: ACPI: DMAR not present May 18 09:23:06 ALPL202 kernel: PCI-GART: No AMD northbridge found. May 18 09:23:06 ALPL202 kernel: system 00:01: iomem range 0xfed1c000-0xfed1ffff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:06: ioport range 0xa10-0xa1f has been reserved May 18 09:23:06 ALPL202 kernel: system 00:07: iomem range 0x400-0x4ff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:07: iomem range 0xfed1c000-0xfed1ffff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:07: iomem range 0xfed20000-0xfed3ffff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:07: iomem range 0xfed40000-0xfed8ffff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:09: iomem range 0xfec00000-0xfec00fff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:09: iomem range 0xfee00000-0xfee00fff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:0c: iomem range 0xe0000000-0xefffffff has been reserved May 18 09:23:06 ALPL202 kernel: system 00:0d: iomem range 0xc0000-0xcffff could not be reserved May 18 09:23:06 ALPL202 kernel: system 00:0d: iomem range 0xe0000-0xfffff could not be reserved May 18 09:23:06 ALPL202 rpc.statd[4138]: Version 1.0.9 Starting May 18 09:23:06 ALPL202 kernel: system 00:0d: iomem range 0xfed90000-0xffffffff could not be reserved May 18 09:23:06 ALPL202 kernel: pci 0000:00:01.0: PCI bridge to [bus 01-01] May 18 09:23:06 ALPL202 kernel: pci 0000:00:01.0: bridge window [io disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:01.0: bridge window [mem disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:01.0: bridge window [mem pref disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:03.0: PCI bridge to [bus 02-02] May 18 09:23:06 ALPL202 kernel: pci 0000:00:03.0: bridge window [io disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:03.0: bridge window [mem disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:03.0: bridge window [mem pref disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:05.0: PCI bridge to [bus 03-03] May 18 09:23:06 ALPL202 kernel: pci 0000:00:05.0: bridge window [io disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:05.0: bridge window [mem 0xfaa00000-0xfaafffff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:05.0: bridge window [mem 0xf8000000-0xf87fffff 64bit pref] May 18 09:23:06 ALPL202 kernel: pci 0000:00:07.0: PCI bridge to [bus 04-04] May 18 09:23:06 ALPL202 kernel: pci 0000:00:07.0: bridge window [io disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:07.0: bridge window [mem 0xfab00000-0xfabfffff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:07.0: bridge window [mem 0xf8800000-0xf8ffffff 64bit pref] May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.0: BAR 12: assigned [mem 0xfae00000-0xfae1ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.0: BAR 12: set to [mem 0xfae00000-0xfae1ffff 64bit] (PCI address [0xfae00000-0xfae1ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.0: BAR 15: assigned [mem 0xfae20000-0xfae3ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.0: BAR 15: set to [mem 0xfae20000-0xfae3ffff 64bit] (PCI address [0xfae20000-0xfae3ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.1: BAR 12: assigned [mem 0xfae40000-0xfae5ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.1: BAR 12: set to [mem 0xfae40000-0xfae5ffff 64bit] (PCI address [0xfae40000-0xfae5ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.1: BAR 15: assigned [mem 0xfae60000-0xfae7ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:07:00.1: BAR 15: set to [mem 0xfae60000-0xfae7ffff 64bit] (PCI address [0xfae60000-0xfae7ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.0: PCI bridge to [bus 07-08] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.0: bridge window [io 0xe000-0xefff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.0: bridge window [mem 0xfae00000-0xfaffffff] May 18 09:23:06 ALPL202 rpc.statd[4138]: statd running as root. chown /var/lib/nfs/statd/sm to choose different user May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.0: bridge window [mem pref disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.0: BAR 12: assigned [mem 0xfac00000-0xfac1ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.0: BAR 12: set to [mem 0xfac00000-0xfac1ffff 64bit] (PCI address [0xfac00000-0xfac1ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.0: BAR 15: assigned [mem 0xfac20000-0xfac3ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.0: BAR 15: set to [mem 0xfac20000-0xfac3ffff 64bit] (PCI address [0xfac20000-0xfac3ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.1: BAR 12: assigned [mem 0xfac40000-0xfac5ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.1: BAR 12: set to [mem 0xfac40000-0xfac5ffff 64bit] (PCI address [0xfac40000-0xfac5ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.1: BAR 15: assigned [mem 0xfac60000-0xfac7ffff 64bit] May 18 09:23:06 ALPL202 kernel: pci 0000:05:00.1: BAR 15: set to [mem 0xfac60000-0xfac7ffff 64bit] (PCI address [0xfac60000-0xfac7ffff]) May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.4: PCI bridge to [bus 05-06] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.4: bridge window [io 0xd000-0xdfff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.4: bridge window [mem 0xfac00000-0xfadfffff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1c.4: bridge window [mem pref disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1e.0: PCI bridge to [bus 09-09] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1e.0: bridge window [io disabled] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1e.0: bridge window [mem 0xfb000000-0xfbefffff] May 18 09:23:06 ALPL202 kernel: pci 0000:00:1e.0: bridge window [mem 0xf9000000-0xf9ffffff 64bit pref] May 18 09:23:06 ALPL202 kernel: GSI 16 sharing vector 0xA9 and IRQ 16 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 16 (level, low) -> IRQ 169 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 16 (level, low) -> IRQ 169 May 18 09:23:06 ALPL202 kernel: NET: Registered protocol family 2 May 18 09:23:06 ALPL202 kernel: IP route cache hash table entries: 524288 (order: 10, 4194304 bytes) May 18 09:23:06 ALPL202 kernel: TCP established hash table entries: 262144 (order: 10, 4194304 bytes) May 18 09:23:06 ALPL202 kernel: TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) May 18 09:23:06 ALPL202 kernel: TCP: Hash tables configured (established 262144 bind 65536) May 18 09:23:06 ALPL202 kernel: TCP reno registered May 18 09:23:06 ALPL202 kernel: audit: initializing netlink socket (disabled) May 18 09:23:06 ALPL202 kernel: type=2000 audit(1431912107.672:1): initialized May 18 09:23:06 ALPL202 kernel: Total HugeTLB memory allocated, 0 May 18 09:23:06 ALPL202 kernel: VFS: Disk quotas dquot_6.5.1 May 18 09:23:06 ALPL202 kernel: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) May 18 09:23:06 ALPL202 kernel: Initializing Cryptographic API May 18 09:23:06 ALPL202 kernel: alg: No test for crc32c (crc32c-generic) May 18 09:23:06 ALPL202 kernel: ksign: Installing public key data May 18 09:23:06 ALPL202 kernel: Loading keyring May 18 09:23:06 ALPL202 kernel: io scheduler noop registered May 18 09:23:06 ALPL202 kernel: io scheduler anticipatory registered May 18 09:23:06 ALPL202 kernel: io scheduler deadline registered (default) May 18 09:23:06 ALPL202 kernel: io scheduler cfq registered May 18 09:23:06 ALPL202 kernel: pci_hotplug: PCI Hot Plug PCI Core version: 0.5 May 18 09:23:06 ALPL202 kernel: ACPI (exconfig-0456): Dynamic SSDT Load - OemId [DpgPmm] OemTableId [ P001Ist] [20060707] May 18 09:23:06 ALPL202 kernel: Real Time Clock Driver v1.12ac May 18 09:23:06 ALPL202 kernel: Non-volatile memory driver v1.2 May 18 09:23:06 ALPL202 kernel: Linux agpgart interface v0.101 (c) Dave Jones May 18 09:23:06 ALPL202 kernel: Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled May 18 09:23:06 ALPL202 kernel: serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A May 18 09:23:06 ALPL202 kernel: serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A May 18 09:23:06 ALPL202 kernel: 00:0a: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A May 18 09:23:06 ALPL202 kernel: 00:0b: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A May 18 09:23:06 ALPL202 kernel: brd: module loaded May 18 09:23:06 ALPL202 kernel: Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 May 18 09:23:06 ALPL202 kernel: ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx May 18 09:23:06 ALPL202 kernel: ide-floppy driver 0.99.newide May 18 09:23:06 ALPL202 kernel: usbcore: registered new driver hiddev May 18 09:23:06 ALPL202 kernel: usbcore: registered new driver usbhid May 18 09:23:06 ALPL202 kernel: drivers/usb/input/hid-core.c: v2.6:USB HID core driver May 18 09:23:06 ALPL202 kernel: PNP: No PS/2 controller found. Probing ports directly. May 18 09:23:06 ALPL202 kernel: serio: i8042 KBD port at 0x60,0x64 irq 1 May 18 09:23:06 ALPL202 kernel: serio: i8042 AUX port at 0x60,0x64 irq 12 May 18 09:23:06 ALPL202 kernel: mice: PS/2 mouse device common for all mice May 18 09:23:06 ALPL202 kernel: md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27 May 18 09:23:06 ALPL202 kernel: md: bitmap version 4.39 May 18 09:23:06 ALPL202 kernel: TCP bic registered May 18 09:23:06 ALPL202 kernel: Initializing IPsec netlink socket May 18 09:23:06 ALPL202 kernel: NET: Registered protocol family 1 May 18 09:23:06 ALPL202 kernel: NET: Registered protocol family 17 May 18 09:23:06 ALPL202 kernel: ACPI: (supports S0 S1 S4 S5) May 18 09:23:06 ALPL202 kernel: Initalizing network drop monitor service May 18 09:23:06 ALPL202 kernel: Freeing unused kernel memory: 224k freed May 18 09:23:06 ALPL202 kernel: Write protecting the kernel read-only data: 533k May 18 09:23:06 ALPL202 kernel: GSI 17 sharing vector 0xE9 and IRQ 17 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1a.7[C] -> GSI 18 (level, low) -> IRQ 233 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1a.7: EHCI Host Controller May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1a.7: new USB bus registered, assigned bus number 1 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1a.7: debug port 1 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1a.7: irq 233, io mem 0xfa9de000 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1a.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004 May 18 09:23:06 ALPL202 kernel: usb usb1: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 1-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 1-0:1.0: 6 ports detected May 18 09:23:06 ALPL202 kernel: GSI 18 sharing vector 0x32 and IRQ 18 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 23 (level, low) -> IRQ 50 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1d.7: EHCI Host Controller May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 2 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1d.7: debug port 1 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1d.7: irq 50, io mem 0xfa9dc000 May 18 09:23:06 ALPL202 kernel: ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004 May 18 09:23:06 ALPL202 kernel: usb usb2: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 2-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 2-0:1.0: 6 ports detected May 18 09:23:06 ALPL202 kernel: USB Universal Host Controller Interface driver v3.0 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1a.0[A] -> GSI 16 (level, low) -> IRQ 169 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.0: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 3 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.0: irq 169, io base 0x0000cc00 May 18 09:23:06 ALPL202 kernel: usb usb3: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 3-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 3-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: GSI 19 sharing vector 0x3A and IRQ 19 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1a.1[B] -> GSI 21 (level, low) -> IRQ 58 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.1: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.1: new USB bus registered, assigned bus number 4 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.1: irq 58, io base 0x0000c880 May 18 09:23:06 ALPL202 kernel: usb usb4: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 4-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 4-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: GSI 20 sharing vector 0x42 and IRQ 20 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1a.2[D] -> GSI 19 (level, low) -> IRQ 66 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.2: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.2: new USB bus registered, assigned bus number 5 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1a.2: irq 66, io base 0x0000c800 May 18 09:23:06 ALPL202 kernel: usb usb5: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 5-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 5-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1d.0[A] -> GSI 23 (level, low) -> IRQ 50 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.0: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 6 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.0: irq 50, io base 0x0000c480 May 18 09:23:06 ALPL202 kernel: usb usb6: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 6-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 6-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1d.1[B] -> GSI 19 (level, low) -> IRQ 66 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.1: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 7 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.1: irq 66, io base 0x0000c400 May 18 09:23:06 ALPL202 kernel: usb usb7: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 7-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 7-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: usb 4-1: new full speed USB device using uhci_hcd and address 2 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1d.2[C] -> GSI 18 (level, low) -> IRQ 233 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.2: UHCI Host Controller May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 8 May 18 09:23:06 ALPL202 kernel: uhci_hcd 0000:00:1d.2: irq 233, io base 0x0000c080 May 18 09:23:06 ALPL202 kernel: usb usb8: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: hub 8-0:1.0: USB hub found May 18 09:23:06 ALPL202 kernel: hub 8-0:1.0: 2 ports detected May 18 09:23:06 ALPL202 kernel: usb 4-1: configuration #1 chosen from 1 choice May 18 09:23:06 ALPL202 kernel: input: Winbond Electronics Corp Hermon USB hidmouse Device as /class/input/input0 May 18 09:23:06 ALPL202 kernel: input,hidraw0: USB HID v1.00 Mouse [Winbond Electronics Corp Hermon USB hidmouse Device] on usb-0000:00:1a.1-1 May 18 09:23:06 ALPL202 kernel: input: Winbond Electronics Corp Hermon USB hidmouse Device as /class/input/input1 May 18 09:23:06 ALPL202 kernel: input,hidraw0: USB HID v1.00 Keyboard [Winbond Electronics Corp Hermon USB hidmouse Device] on usb-0000:00:1a.1-1 May 18 09:23:06 ALPL202 kernel: md: raid1 personality registered for level 1 May 18 09:23:06 ALPL202 kernel: SCSI subsystem initialized May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 66 May 18 09:23:06 ALPL202 kernel: ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 0x3f impl SATA mode May 18 09:23:06 ALPL202 kernel: ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo pio slum part ems May 18 09:23:06 ALPL202 kernel: scsi0 : ahci May 18 09:23:06 ALPL202 kernel: scsi1 : ahci May 18 09:23:06 ALPL202 kernel: scsi2 : ahci May 18 09:23:06 ALPL202 kernel: scsi3 : ahci May 18 09:23:06 ALPL202 kernel: scsi4 : ahci May 18 09:23:06 ALPL202 kernel: scsi5 : ahci May 18 09:23:06 ALPL202 kernel: ata1: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da100 irq 74 May 18 09:23:06 ALPL202 kernel: ata2: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da180 irq 74 May 18 09:23:06 ALPL202 kernel: ata3: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da200 irq 74 May 18 09:23:06 ALPL202 kernel: ata4: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da280 irq 74 May 18 09:23:06 ALPL202 kernel: ata5: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da300 irq 74 May 18 09:23:06 ALPL202 kernel: ata6: SATA max UDMA/133 abar m2048@0xfa9da000 port 0xfa9da380 irq 74 May 18 09:23:06 ALPL202 kernel: ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 18 09:23:06 ALPL202 kernel: ata1.00: ATA-8: ST9500530NS, SN04, max UDMA/133 May 18 09:23:06 ALPL202 kernel: ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32) May 18 09:23:06 ALPL202 kernel: ata1.00: configured for UDMA/133 May 18 09:23:06 ALPL202 kernel: ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300) May 18 09:23:06 ALPL202 kernel: ata2.00: ATA-8: ST9500530NS, SN04, max UDMA/133 May 18 09:23:06 ALPL202 kernel: ata2.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32) May 18 09:23:06 ALPL202 kernel: ata2.00: configured for UDMA/133 May 18 09:23:06 ALPL202 kernel: ata3: SATA link down (SStatus 0 SControl 300) May 18 09:23:06 ALPL202 kernel: ata4: SATA link down (SStatus 0 SControl 300) May 18 09:23:06 ALPL202 kernel: ata5: SATA link down (SStatus 0 SControl 300) May 18 09:23:06 ALPL202 kernel: ata6: SATA link down (SStatus 0 SControl 300) May 18 09:23:06 ALPL202 kernel: Vendor: ATA Model: ST9500530NS Rev: SN04 May 18 09:23:06 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:23:06 ALPL202 kernel: SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB) May 18 09:23:06 ALPL202 kernel: sda: Write Protect is off May 18 09:23:06 ALPL202 kernel: SCSI device sda: drive cache: write back May 18 09:23:06 ALPL202 kernel: SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB) May 18 09:23:06 ALPL202 kernel: sda: Write Protect is off May 18 09:23:06 ALPL202 kernel: SCSI device sda: drive cache: write back May 18 09:23:06 ALPL202 kernel: sda: sda1 sda2 sda3 May 18 09:23:06 ALPL202 kernel: sd 0:0:0:0: Attached scsi disk sda May 18 09:23:06 ALPL202 kernel: Vendor: ATA Model: ST9500530NS Rev: SN04 May 18 09:23:06 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:23:06 ALPL202 kernel: SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB) May 18 09:23:06 ALPL202 kernel: sdb: Write Protect is off May 18 09:23:06 ALPL202 kernel: SCSI device sdb: drive cache: write back May 18 09:23:06 ALPL202 kernel: SCSI device sdb: 976773168 512-byte hdwr sectors (500108 MB) May 18 09:23:06 ALPL202 kernel: sdb: Write Protect is off May 18 09:23:06 ALPL202 kernel: SCSI device sdb: drive cache: write back May 18 09:23:06 ALPL202 kernel: sdb: sdb1 sdb2 sdb3 May 18 09:23:06 ALPL202 kernel: sd 1:0:0:0: Attached scsi disk sdb May 18 09:23:06 ALPL202 kernel: Initializing USB Mass Storage driver... May 18 09:23:06 ALPL202 kernel: usbcore: registered new driver usb-storage May 18 09:23:06 ALPL202 kernel: USB Mass Storage support registered. May 18 09:23:06 ALPL202 kernel: device-mapper: uevent: version 1.0.3 May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: 4.11.6-ioctl (2011-02-18) initialised: dm-devel@redhat.com May 18 09:23:06 ALPL202 kernel: device-mapper: dm-raid45: initialized v0.2594l May 18 09:23:06 ALPL202 kernel: md: Autodetecting RAID arrays. May 18 09:23:06 ALPL202 kernel: md: autorun ... May 18 09:23:06 ALPL202 kernel: md: considering sdb3 ... May 18 09:23:06 ALPL202 kernel: md: adding sdb3 ... May 18 09:23:06 ALPL202 kernel: md: sdb2 has different UUID to sdb3 May 18 09:23:06 ALPL202 kernel: md: sdb1 has different UUID to sdb3 May 18 09:23:06 ALPL202 kernel: md: adding sda3 ... May 18 09:23:06 ALPL202 kernel: md: sda2 has different UUID to sdb3 May 18 09:23:06 ALPL202 kernel: md: sda1 has different UUID to sdb3 May 18 09:23:06 ALPL202 kernel: md: created md2 May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: running: May 18 09:23:06 ALPL202 kernel: raid1: raid set md2 active with 2 out of 2 mirrors May 18 09:23:06 ALPL202 kernel: md: considering sdb2 ... May 18 09:23:06 ALPL202 kernel: md: adding sdb2 ... May 18 09:23:06 ALPL202 kernel: md: sdb1 has different UUID to sdb2 May 18 09:23:06 ALPL202 kernel: md: adding sda2 ... May 18 09:23:06 ALPL202 kernel: md: sda1 has different UUID to sdb2 May 18 09:23:06 ALPL202 kernel: md: created md1 May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: running: May 18 09:23:06 ALPL202 kernel: raid1: raid set md1 active with 2 out of 2 mirrors May 18 09:23:06 ALPL202 kernel: md: considering sdb1 ... May 18 09:23:06 ALPL202 kernel: md: adding sdb1 ... May 18 09:23:06 ALPL202 kernel: md: adding sda1 ... May 18 09:23:06 ALPL202 kernel: md: created md0 May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: bind May 18 09:23:06 ALPL202 kernel: md: running: May 18 09:23:06 ALPL202 kernel: raid1: raid set md0 active with 2 out of 2 mirrors May 18 09:23:06 ALPL202 kernel: md: ... autorun DONE. May 18 09:23:06 ALPL202 kernel: md: Autodetecting RAID arrays. May 18 09:23:06 ALPL202 kernel: md: autorun ... May 18 09:23:06 ALPL202 kernel: md: ... autorun DONE. May 18 09:23:06 ALPL202 kernel: EXT3-fs: INFO: recovery required on readonly filesystem. May 18 09:23:06 ALPL202 kernel: EXT3-fs: write access will be enabled during recovery. May 18 09:23:06 ALPL202 kernel: kjournald starting. Commit interval 5 seconds May 18 09:23:06 ALPL202 kernel: EXT3-fs: md1: orphan cleanup on readonly fs May 18 09:23:06 ALPL202 kernel: EXT3-fs: md1: 1 orphan inode deleted May 18 09:23:06 ALPL202 kernel: EXT3-fs: recovery complete. May 18 09:23:06 ALPL202 kernel: EXT3-fs: mounted filesystem with ordered data mode. May 18 09:23:06 ALPL202 kernel: SELinux: Disabled at runtime. May 18 09:23:06 ALPL202 kernel: type=1404 audit(1431912143.440:2): selinux=0 auid=4294967295 ses=4294967295 May 18 09:23:06 ALPL202 kernel: input: PC Speaker as /class/input/input2 May 18 09:23:06 ALPL202 kernel: dca service started, version 1.8 May 18 09:23:06 ALPL202 kernel: shpchp: Standard Hot Plug PCI Controller Driver version: 0.4 May 18 09:23:06 ALPL202 kernel: EDAC MC: Ver: 2.0.1 Feb 12 2015 May 18 09:23:06 ALPL202 kernel: 802.1Q VLAN Support v1.8 Ben Greear May 18 09:23:06 ALPL202 kernel: All bugs added by David S. Miller May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:00:1f.3[C] -> GSI 18 (level, low) -> IRQ 233 May 18 09:23:06 ALPL202 kernel: igb: Intel(R) Gigabit Ethernet Network Driver - version 4.0.1-k1-1 May 18 09:23:06 ALPL202 kernel: igb: Copyright (c) 2007-2012 Intel Corporation. May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:07:00.0[A] -> GSI 16 (level, low) -> IRQ 169 May 18 09:23:06 ALPL202 kernel: sd 0:0:0:0: Attached scsi generic sg0 type 0 May 18 09:23:06 ALPL202 kernel: sd 1:0:0:0: Attached scsi generic sg1 type 0 May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.0: Intel(R) Gigabit Ethernet Network Connection May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 00:25:90:14:fa:ec May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.0: eth0: PBA No: FFFFFF-0FF May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.0: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s) May 18 09:23:06 ALPL202 kernel: GSI 21 sharing vector 0x82 and IRQ 21 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:07:00.1[B] -> GSI 17 (level, low) -> IRQ 130 May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.1: Intel(R) Gigabit Ethernet Network Connection May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 00:25:90:14:fa:ed May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.1: eth1: PBA No: FFFFFF-0FF May 18 09:23:06 ALPL202 kernel: igb 0000:07:00.1: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:05:00.0[A] -> GSI 16 (level, low) -> IRQ 169 May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.0: Intel(R) Gigabit Ethernet Network Connection May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.0: eth2: (PCIe:2.5Gb/s:Width x2) 00:25:90:14:fa:ee May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.0: eth2: PBA No: FFFFFF-0FF May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.0: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s) May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:05:00.1[B] -> GSI 17 (level, low) -> IRQ 130 May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.1: Intel(R) Gigabit Ethernet Network Connection May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.1: eth3: (PCIe:2.5Gb/s:Width x2) 00:25:90:14:fa:ef May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.1: eth3: PBA No: FFFFFF-0FF May 18 09:23:06 ALPL202 kernel: igb 0000:05:00.1: Using MSI-X interrupts. 4 rx queue(s), 1 tx queue(s) May 18 09:23:06 ALPL202 kernel: mlx4_core: Mellanox ConnectX core driver v1.0-mlnx_ofed1.5.3 (November 3, 2011) May 18 09:23:06 ALPL202 kernel: mlx4_core: Initializing 0000:03:00.0 May 18 09:23:06 ALPL202 kernel: GSI 22 sharing vector 0x5B and IRQ 22 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 26 (level, low) -> IRQ 91 May 18 09:23:06 ALPL202 kernel: mlx4_core: Initializing 0000:04:00.0 May 18 09:23:06 ALPL202 kernel: GSI 23 sharing vector 0x4C and IRQ 23 May 18 09:23:06 ALPL202 kernel: ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 30 (level, low) -> IRQ 76 May 18 09:23:06 ALPL202 kernel: mlx4_en: Mellanox ConnectX HCA Ethernet driver v1.5.10 (Jan 2013) May 18 09:23:06 ALPL202 kernel: floppy0: no floppy controllers found May 18 09:23:06 ALPL202 kernel: work still pending May 18 09:23:06 ALPL202 kernel: lp: driver loaded but no devices found May 18 09:23:06 ALPL202 kernel: NET: Registered protocol family 10 May 18 09:23:06 ALPL202 kernel: lo: Disabled Privacy Extensions May 18 09:23:06 ALPL202 kernel: IPv6 over IPv4 tunneling driver May 18 09:23:06 ALPL202 kernel: sit0: Disabled Privacy Extensions May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9000000,1000000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9fe0000,10000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9fc0000,20000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9f80000,40000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9f00000,80000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9e00000,100000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9c00000,200000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9800000,400000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9fe0000,10000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9fc0000,20000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9f80000,40000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9f00000,80000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9e00000,100000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9c00000,200000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9800000,400000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:23:06 ALPL202 kernel: ACPI: Power Button (FF) [PWRF] May 18 09:23:06 ALPL202 kernel: ACPI: Power Button (CM) [PWRB] May 18 09:23:06 ALPL202 kernel: ACPI: Mapper loaded May 18 09:23:06 ALPL202 kernel: dell-wmi: No known WMI GUID found May 18 09:23:06 ALPL202 kernel: md: Autodetecting RAID arrays. May 18 09:23:06 ALPL202 kernel: md: autorun ... May 18 09:23:06 ALPL202 kernel: md: ... autorun DONE. May 18 09:23:06 ALPL202 kernel: device-mapper: multipath: version 1.0.6 loaded May 18 09:23:06 ALPL202 kernel: device-mapper: multipath round-robin: version 1.0.0 loaded May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: EXT3 FS on md1, internal journal May 18 09:23:06 ALPL202 kernel: kjournald starting. Commit interval 5 seconds May 18 09:23:06 ALPL202 kernel: EXT3 FS on md0, internal journal May 18 09:23:06 ALPL202 kernel: EXT3-fs: mounted filesystem with ordered data mode. May 18 09:23:06 ALPL202 kernel: Adding 4192888k swap on /dev/md2. Priority:-1 extents:1 across:4192888k May 18 09:23:06 ALPL202 kernel: IA-32 Microcode Update Driver: v1.14a May 18 09:23:06 ALPL202 kernel: mlx4_ib: Mellanox ConnectX InfiniBand driver v1.0-mlnx_ofed1.5.3 (November 3, 2011) May 18 09:23:06 ALPL202 kernel: Default coalesing params for mtu:2044 - rx_frames:88 rx_usecs:16 May 18 09:23:06 ALPL202 last message repeated 2 times May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_UP): ib0: link is not ready May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_CHANGE): ib0: link becomes ready May 18 09:23:06 ALPL202 kernel: ib0: enabling connected mode will cause multicast packet drops May 18 09:23:06 ALPL202 kernel: ib0: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: ib0: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: ib1: enabling connected mode will cause multicast packet drops May 18 09:23:06 ALPL202 kernel: ib1: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: ib1: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: ib2: enabling connected mode will cause multicast packet drops May 18 09:23:06 ALPL202 kernel: ib2: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: ib2: mtu > 2044 will cause multicast packet drops. May 18 09:23:06 ALPL202 kernel: 8021q: adding VLAN 0 to HW filter on device eth0 May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_UP): eth0: link is not ready May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: device-mapper: table: 253:0: multipath: error getting device May 18 09:23:06 ALPL202 kernel: device-mapper: ioctl: error adding target to table May 18 09:23:06 ALPL202 kernel: cm_req_handler: no listen_cm_id_priv May 18 09:23:06 ALPL202 last message repeated 31 times May 18 09:23:06 ALPL202 kernel: igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready May 18 09:23:06 ALPL202 kernel: 8021q: adding VLAN 0 to HW filter on device eth2 May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_UP): eth2: link is not ready May 18 09:23:06 ALPL202 kernel: cm_req_handler: no listen_cm_id_priv May 18 09:23:06 ALPL202 last message repeated 25 times May 18 09:23:06 ALPL202 kernel: igb: eth2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX May 18 09:23:06 ALPL202 kernel: ADDRCONF(NETDEV_CHANGE): eth2: link becomes ready May 18 09:23:06 ALPL202 kernel: cm_req_handler: no listen_cm_id_priv May 18 09:23:15 ALPL202 last message repeated 58 times May 18 09:23:16 ALPL202 kernel: cm_dreq_handler: no cm_id_priv May 18 09:23:16 ALPL202 kernel: cm_req_handler: no listen_cm_id_priv May 18 09:23:47 ALPL202 last message repeated 64 times May 18 09:24:49 ALPL202 last message repeated 111 times May 18 09:25:33 ALPL202 last message repeated 66 times May 18 09:28:14 ALPL202 OpenSM[5010]: /var/log/opensm-1.log log file opened May 18 09:28:14 ALPL202 OpenSM[5026]: /var/log/opensm-2.log log file opened May 18 09:28:14 ALPL202 OpenSM[5010]: OpenSM 3.3.13.MLNX_20130110_cd124d3 May 18 09:28:14 ALPL202 OpenSM[5026]: OpenSM 3.3.13.MLNX_20130110_cd124d3 May 18 09:28:14 ALPL202 OpenSM[5010]: Entering DISCOVERING state May 18 09:28:14 ALPL202 OpenSM[5026]: Entering DISCOVERING state May 18 09:28:14 ALPL202 OpenSM[5026]: Entering MASTER state May 18 09:28:14 ALPL202 kernel: ib_srp: ASYNC event= 11 on device= mlx4_1 port= 2 May 18 09:28:14 ALPL202 kernel: ib_srp: ASYNC event= 9 on device= mlx4_1 port= 2 May 18 09:28:14 ALPL202 OpenSM[5010]: Entering MASTER state May 18 09:28:14 ALPL202 kernel: ib_srp: ASYNC event= 11 on device= mlx4_1 port= 1 May 18 09:28:14 ALPL202 kernel: ib_srp: ASYNC event= 9 on device= mlx4_1 port= 1 May 18 09:28:15 ALPL202 run_srp_daemon[5076]: starting srp_daemon: [HCA=mlx4_1] [port=1] May 18 09:28:16 ALPL202 run_srp_daemon[5093]: starting srp_daemon: [HCA=mlx4_1] [port=2] May 18 09:28:18 ALPL202 kernel: scsi6 : SRP.T10:681A0D0003C90200 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sd 6:0:0:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. May 18 09:28:18 ALPL202 kernel: sdc: Unit Not Ready, sense: May 18 09:28:18 ALPL202 kernel: : Current: sense key: Unit Attention May 18 09:28:18 ALPL202 kernel: Add. Sense: Reported luns data has changed May 18 09:28:18 ALPL202 kernel: May 18 09:28:18 ALPL202 kernel: sdc : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdc: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdc: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdc: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 multipathd: sdc: add path (uevent) May 18 09:28:18 ALPL202 kernel: sdc : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdc: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdc: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdc: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdc: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:0: Attached scsi disk sdc May 18 09:28:18 ALPL202 kernel: sd 6:0:0:0: Attached scsi generic sg2 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdd : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdd: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdd: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdd: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdd : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdd: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdd: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdd: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdd: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:1: Attached scsi disk sdd May 18 09:28:18 ALPL202 kernel: sd 6:0:0:1: Attached scsi generic sg3 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sde : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sde: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sde: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sde: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sde : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sde: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sde: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sde: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sde: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:2: Attached scsi disk sde May 18 09:28:18 ALPL202 kernel: sd 6:0:0:2: Attached scsi generic sg4 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdf : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdf: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdf: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdf: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdf : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdf: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdf: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdf: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdf: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:3: Attached scsi disk sdf May 18 09:28:18 ALPL202 kernel: sd 6:0:0:3: Attached scsi generic sg5 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdg : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdg: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdg: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdg: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdg : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdg: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdg: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdg: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdg: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:4: Attached scsi disk sdg May 18 09:28:18 ALPL202 kernel: sd 6:0:0:4: Attached scsi generic sg6 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdh : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdh: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdh: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdh: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdh : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdh: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdh: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdh: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdh: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:5: Attached scsi disk sdh May 18 09:28:18 ALPL202 kernel: sd 6:0:0:5: Attached scsi generic sg7 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdi : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdi: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdi: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdi: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdi : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdi: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdi: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdi: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdi: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:6: Attached scsi disk sdi May 18 09:28:18 ALPL202 kernel: sd 6:0:0:6: Attached scsi generic sg8 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdj : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdj: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdj: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdj: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdj : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdj: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdj: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdj: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdj: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:7: Attached scsi disk sdj May 18 09:28:18 ALPL202 kernel: sd 6:0:0:7: Attached scsi generic sg9 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdk : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdk: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdk: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdk: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdk : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdk: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdk: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdk: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdk: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:8: Attached scsi disk sdk May 18 09:28:18 ALPL202 kernel: sd 6:0:0:8: Attached scsi generic sg10 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdl : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdl: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdl: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdl: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdl : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdl: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdl: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdl: drive cache: write through w/ FUA May 18 09:28:18 ALPL202 kernel: sdl: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 6:0:0:9: Attached scsi disk sdl May 18 09:28:18 ALPL202 kernel: sd 6:0:0:9: Attached scsi generic sg11 type 0 May 18 09:28:18 ALPL202 kernel: scsi7 : SRP.T10:5A3C0D0003C90200 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sd 7:0:0:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatically remap LUN assignments. May 18 09:28:18 ALPL202 kernel: sdm: Unit Not Ready, sense: May 18 09:28:18 ALPL202 kernel: : Current: sense key: Unit Attention May 18 09:28:18 ALPL202 kernel: Add. Sense: Reported luns data has changed May 18 09:28:18 ALPL202 kernel: May 18 09:28:18 ALPL202 kernel: sdm : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdm: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdm: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdm: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdm : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdm: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdm: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdm: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdm: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:0: Attached scsi disk sdm May 18 09:28:18 ALPL202 kernel: sd 7:0:0:0: Attached scsi generic sg12 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdn : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdn: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdn: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdn: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdn : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdn: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdn: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdn: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdn: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:1: Attached scsi disk sdn May 18 09:28:18 ALPL202 kernel: sd 7:0:0:1: Attached scsi generic sg13 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdo : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdo: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdo: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdo: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdo : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdo: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdo: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdo: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdo: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:2: Attached scsi disk sdo May 18 09:28:18 ALPL202 kernel: sd 7:0:0:2: Attached scsi generic sg14 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdp : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdp: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdp: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdp: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdp : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdp: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdp: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdp: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdp: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:3: Attached scsi disk sdp May 18 09:28:18 ALPL202 kernel: sd 7:0:0:3: Attached scsi generic sg15 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdq : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdq: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdq: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdq: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdq : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdq: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdq: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdq: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdq: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:4: Attached scsi disk sdq May 18 09:28:18 ALPL202 kernel: sd 7:0:0:4: Attached scsi generic sg16 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdr : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdr: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdr: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdr: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdr : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdr: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdr: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdr: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdr: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:5: Attached scsi disk sdr May 18 09:28:18 ALPL202 kernel: sd 7:0:0:5: Attached scsi generic sg17 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sds : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sds: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sds: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sds: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sds : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sds: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sds: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sds: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sds: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:6: Attached scsi disk sds May 18 09:28:18 ALPL202 kernel: sd 7:0:0:6: Attached scsi generic sg18 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdt : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdt: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdt: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdt: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdt : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdt: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdt: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdt: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdt: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:7: Attached scsi disk sdt May 18 09:28:18 ALPL202 kernel: sd 7:0:0:7: Attached scsi generic sg19 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdu : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdu: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdu: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdu: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdu : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdu: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdu: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdu: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdu: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:8: Attached scsi disk sdu May 18 09:28:18 ALPL202 kernel: sd 7:0:0:8: Attached scsi generic sg20 type 0 May 18 09:28:18 ALPL202 kernel: Vendor: DDN Model: SFA 10000 Rev: 1.03 May 18 09:28:18 ALPL202 kernel: Type: Direct-Access ANSI SCSI revision: 05 May 18 09:28:18 ALPL202 kernel: sdv : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdv: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdv: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdv: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdv : very big device. try to use READ CAPACITY(16). May 18 09:28:18 ALPL202 kernel: SCSI device sdv: 4412407808 512-byte hdwr sectors (2259153 MB) May 18 09:28:18 ALPL202 kernel: sdv: Write Protect is off May 18 09:28:18 ALPL202 kernel: SCSI device sdv: drive cache: write back w/ FUA May 18 09:28:18 ALPL202 kernel: sdv: unknown partition table May 18 09:28:18 ALPL202 kernel: sd 7:0:0:9: Attached scsi disk sdv May 18 09:28:18 ALPL202 kernel: sd 7:0:0:9: Attached scsi generic sg21 type 0 May 18 09:28:18 ALPL202 multipathd: OST00: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:32 10] May 18 09:28:18 ALPL202 multipathd: OST00: event checker started May 18 09:28:18 ALPL202 multipathd: sdd: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST01: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:48 10] May 18 09:28:18 ALPL202 multipathd: OST01: event checker started May 18 09:28:18 ALPL202 multipathd: sde: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST02: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:64 10] May 18 09:28:18 ALPL202 multipathd: OST02: event checker started May 18 09:28:18 ALPL202 multipathd: sdf: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST03: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:80 10] May 18 09:28:18 ALPL202 multipathd: OST03: event checker started May 18 09:28:18 ALPL202 multipathd: sdg: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST04: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:96 10] May 18 09:28:18 ALPL202 multipathd: OST04: event checker started May 18 09:28:18 ALPL202 multipathd: sdh: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST05: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:112 10] May 18 09:28:18 ALPL202 multipathd: OST05: event checker started May 18 09:28:18 ALPL202 multipathd: sdi: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST06: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:128 10] May 18 09:28:18 ALPL202 multipathd: OST06: event checker started May 18 09:28:18 ALPL202 multipathd: sdj: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST07: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:144 10] May 18 09:28:18 ALPL202 multipathd: OST07: event checker started May 18 09:28:18 ALPL202 multipathd: sdk: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST08: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:160 10] May 18 09:28:18 ALPL202 multipathd: OST08: event checker started May 18 09:28:18 ALPL202 multipathd: sdl: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST09: load table [0 4412407808 multipath 0 0 1 1 round-robin 0 1 1 8:176 10] May 18 09:28:18 ALPL202 multipathd: OST09: event checker started May 18 09:28:18 ALPL202 multipathd: sdm: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST00: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 8:32 10 round-robin 0 1 1 8:192 10] May 18 09:28:18 ALPL202 multipathd: sdn: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST01: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 8:48 10 round-robin 0 1 1 8:208 10] May 18 09:28:18 ALPL202 multipathd: sdo: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST02: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 8:64 10 round-robin 0 1 1 8:224 10] May 18 09:28:18 ALPL202 multipathd: sdp: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST03: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 8:80 10 round-robin 0 1 1 8:240 10] May 18 09:28:18 ALPL202 multipathd: sdq: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST04: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 8:96 10 round-robin 0 1 1 65:0 10] May 18 09:28:18 ALPL202 multipathd: sdr: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST05: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 65:16 10 round-robin 0 1 1 8:112 10] May 18 09:28:18 ALPL202 multipathd: sds: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST06: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 65:32 10 round-robin 0 1 1 8:128 10] May 18 09:28:18 ALPL202 multipathd: sdt: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST07: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 65:48 10 round-robin 0 1 1 8:144 10] May 18 09:28:18 ALPL202 multipathd: sdu: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST08: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 65:64 10 round-robin 0 1 1 8:160 10] May 18 09:28:18 ALPL202 multipathd: sdv: add path (uevent) May 18 09:28:18 ALPL202 multipathd: OST09: load table [0 4412407808 multipath 0 0 2 1 round-robin 0 1 1 65:80 10 round-robin 0 1 1 8:176 10] May 18 09:28:18 ALPL202 multipathd: dm-0: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-0: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-1: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-1: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-2: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-2: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-3: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-3: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-4: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-4: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-5: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-5: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-6: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-6: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-7: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-7: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-8: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-8: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-9: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-9: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-0: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-0: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-1: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-1: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-2: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-2: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-3: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-3: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-4: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-4: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-5: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-5: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-6: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-6: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-7: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-7: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-8: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-8: devmap already registered May 18 09:28:18 ALPL202 multipathd: dm-9: add map (uevent) May 18 09:28:18 ALPL202 multipathd: dm-9: devmap already registered May 18 09:28:19 ALPL202 automount[6224]: lookup_read_master: lookup(nisplus): couldn't locate nis+ table auto.master May 18 09:28:19 ALPL202 ntpd[6282]: ntpd 4.2.2p1@1.1570-o Sat Dec 19 00:56:13 UTC 2009 (1) May 18 09:28:19 ALPL202 ntpd[6285]: precision = 1.000 usec May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface wildcard, 0.0.0.0#123 Disabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface wildcard, ::#123 Disabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface lo, ::1#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface eth2, fe80::225:90ff:fe14:faee#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface ib0, fe80::225:90ff:ff07:1411#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface eth0, fe80::225:90ff:fe14:faec#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface lo, 127.0.0.1#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface eth0, 10.2.2.69#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface eth2, 192.168.2.69#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: Listening on interface ib0, 10.3.2.69#123 Enabled May 18 09:28:19 ALPL202 ntpd[6285]: kernel time sync status 0040 May 18 09:28:19 ALPL202 xinetd[6267]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. May 18 09:28:19 ALPL202 xinetd[6267]: Started working: 0 available services May 18 09:28:19 ALPL202 ntpd[6285]: frequency initialized -32.849 PPM from /var/lib/ntp/drift May 18 09:28:19 ALPL202 ntpd[6285]: can't open key file /etc/ntp/keys: No such file or directory May 18 09:28:21 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:28:21 ALPL202 kernel: mtrr: type mismatch for f9000000,1000000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9fe0000,10000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9fc0000,20000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9f80000,40000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9f00000,80000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9e00000,100000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9c00000,200000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9800000,400000 old: write-back new: write-combining May 18 09:28:22 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9fe0000,10000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9fc0000,20000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9f80000,40000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9f00000,80000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9e00000,100000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9c00000,200000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9800000,400000 old: write-back new: write-combining May 18 09:28:23 ALPL202 kernel: mtrr: type mismatch for f9000000,800000 old: write-back new: write-combining May 18 09:31:35 ALPL202 ntpd[6285]: synchronized to LOCAL(0), stratum 10 May 18 09:31:35 ALPL202 ntpd[6285]: kernel time sync enabled 0001 May 18 09:31:43 ALPL202 kernel: Lustre: Build Version: 1.8.9.ddn10-gbbc7f89-CHANGED-2.6.18-348.3.1.el5_lustre.1.8.9.ddn8 May 18 09:31:43 ALPL202 kernel: Lustre: Listener bound to ib0:10.3.2.69:987:mlx4_0 May 18 09:31:43 ALPL202 kernel: Lustre: Added LNI 10.3.2.69@o2ib [8/64/0/180] May 18 09:31:43 ALPL202 kernel: Lustre: Lustre Client File System; http://www.lustre.org/ May 18 09:31:44 ALPL202 kernel: init dynlocks cache May 18 09:31:44 ALPL202 kernel: ldiskfs created from ext4-2.6-rhel5 May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-5): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-5): mounted filesystem with ordered data mode May 18 09:31:44 ALPL202 multipathd: dm-5: umount map (uevent) May 18 09:31:44 ALPL202 kernel: JBD: barrier-based sync failed on dm-5-8 - disabling barriers May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-5): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-5): mounted filesystem with ordered data mode May 18 09:31:44 ALPL202 kernel: Lustre: MGC10.3.2.72@o2ib: Reactivating import May 18 09:31:44 ALPL202 kernel: Lustre: Filtering OBD driver; http://wiki.whamcloud.com/ May 18 09:31:44 ALPL202 kernel: Lustre: 7607:0:(filter.c:1003:filter_init_server_data()) RECOVERY: service LFS02-OST0005, 54 recoverable clients, 0 delayed clients, last_rcvd 81604397798 May 18 09:31:44 ALPL202 kernel: JBD: barrier-based sync failed on dm-5-8 - disabling barriers May 18 09:31:44 ALPL202 kernel: Lustre: LFS02-OST0005: Now serving LFS02-OST0005 on /dev/mpath/OST05 with recovery enabled May 18 09:31:44 ALPL202 kernel: Lustre: LFS02-OST0005: Will be in recovery for at least 5:00, or until 54 clients reconnect May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-6): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-6): mounted filesystem with ordered data mode May 18 09:31:44 ALPL202 multipathd: dm-6: umount map (uevent) May 18 09:31:44 ALPL202 kernel: JBD: barrier-based sync failed on dm-6-8 - disabling barriers May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-6): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-6): mounted filesystem with ordered data mode May 18 09:31:44 ALPL202 kernel: Lustre: 7760:0:(filter.c:1003:filter_init_server_data()) RECOVERY: service LFS02-OST0006, 54 recoverable clients, 0 delayed clients, last_rcvd 81604398304 May 18 09:31:44 ALPL202 kernel: JBD: barrier-based sync failed on dm-6-8 - disabling barriers May 18 09:31:44 ALPL202 kernel: Lustre: LFS02-OST0006: Now serving LFS02-OST0006 on /dev/mpath/OST06 with recovery enabled May 18 09:31:44 ALPL202 kernel: Lustre: LFS02-OST0006: Will be in recovery for at least 5:00, or until 54 clients reconnect May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-7): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:44 ALPL202 kernel: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 multipathd: dm-7: umount map (uevent) May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-7-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-7): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-7): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-7-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-8): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 multipathd: dm-8: umount map (uevent) May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-8-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-8): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-8): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-8-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-9): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 multipathd: dm-9: umount map (uevent) May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-9-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-9): warning: maximal mount count reached, running e2fsck is recommended May 18 09:31:45 ALPL202 kernel: LDISKFS-fs (dm-9): mounted filesystem with ordered data mode May 18 09:31:45 ALPL202 kernel: Lustre: 8162:0:(filter.c:1003:filter_init_server_data()) RECOVERY: service LFS02-OST0009, 54 recoverable clients, 0 delayed clients, last_rcvd 81604398247 May 18 09:31:45 ALPL202 kernel: Lustre: 8162:0:(filter.c:1003:filter_init_server_data()) Skipped 2 previous similar messages May 18 09:31:45 ALPL202 kernel: JBD: barrier-based sync failed on dm-9-8 - disabling barriers May 18 09:31:45 ALPL202 kernel: Lustre: LFS02-OST0009: Now serving LFS02-OST0009 on /dev/mpath/OST09 with recovery enabled May 18 09:31:45 ALPL202 kernel: Lustre: Skipped 2 previous similar messages May 18 09:31:45 ALPL202 kernel: Lustre: LFS02-OST0009: Will be in recovery for at least 5:00, or until 54 clients reconnect May 18 09:31:45 ALPL202 kernel: Lustre: Skipped 2 previous similar messages May 18 09:31:48 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:48 ALPL202 kernel: Lustre: 7339:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0005: 53 recoverable clients remain May 18 09:31:48 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:49 ALPL202 last message repeated 3 times May 18 09:31:49 ALPL202 kernel: Lustre: 8534:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0005: 52 recoverable clients remain May 18 09:31:49 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:50 ALPL202 last message repeated 2 times May 18 09:31:50 ALPL202 kernel: Lustre: 8442:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0005: 51 recoverable clients remain May 18 09:31:50 ALPL202 kernel: Lustre: 8442:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 3 previous similar messages May 18 09:31:51 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:52 ALPL202 last message repeated 3 times May 18 09:31:52 ALPL202 kernel: Lustre: 8326:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0008: 53 recoverable clients remain May 18 09:31:52 ALPL202 kernel: Lustre: 8326:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 1 previous similar message May 18 09:31:52 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:53 ALPL202 last message repeated 3 times May 18 09:31:54 ALPL202 kernel: Lustre: 8570:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0009: 50 recoverable clients remain May 18 09:31:54 ALPL202 kernel: Lustre: 8570:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 5 previous similar messages May 18 09:31:54 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:31:58 ALPL202 last message repeated 5 times May 18 09:31:59 ALPL202 kernel: Lustre: 8383:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0006: 47 recoverable clients remain May 18 09:31:59 ALPL202 kernel: Lustre: 8383:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 15 previous similar messages May 18 09:32:00 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:32:06 ALPL202 last message repeated 15 times May 18 09:32:07 ALPL202 kernel: Lustre: 7350:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0006: 30 recoverable clients remain May 18 09:32:07 ALPL202 kernel: Lustre: 7350:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 87 previous similar messages May 18 09:32:07 ALPL202 kernel: ib_send_cm_mra: cm_id_priv->id.state: 0x6 May 18 09:32:19 ALPL202 last message repeated 14 times May 18 09:32:23 ALPL202 kernel: Lustre: 8294:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) LFS02-OST0007: 4 recoverable clients remain May 18 09:32:23 ALPL202 kernel: Lustre: 8294:0:(ldlm_lib.c:1817:target_queue_last_replay_reply()) Skipped 130 previous similar messages May 18 09:32:25 ALPL202 kernel: Lustre: LFS02-OST0009: Recovery period over after 0:37, of 54 clients 54 recovered and 0 were evicted. May 18 09:32:25 ALPL202 kernel: Lustre: LFS02-OST0009: sending delayed replies to recovered clients May 18 09:32:25 ALPL202 kernel: Lustre: LFS02-OST0009: received MDS connection from 10.3.2.72@o2ib May 18 09:32:25 ALPL202 kernel: Lustre: 8504:0:(filter.c:3123:filter_destroy_precreated()) LFS02-OST0009: deleting orphan objects from 266426877 to 266430932, orphan objids won't be reused any more. May 18 09:32:26 ALPL202 kernel: Lustre: LFS02-OST0008: Recovery period over after 0:34, of 54 clients 54 recovered and 0 were evicted. May 18 09:32:26 ALPL202 kernel: Lustre: LFS02-OST0008: sending delayed replies to recovered clients May 18 09:32:26 ALPL202 kernel: Lustre: LFS02-OST0008: received MDS connection from 10.3.2.72@o2ib May 18 09:32:26 ALPL202 kernel: Lustre: 8541:0:(filter.c:3123:filter_destroy_precreated()) LFS02-OST0008: deleting orphan objects from 266242361 to 266246413, orphan objids won't be reused any more. May 18 09:32:27 ALPL202 kernel: Lustre: LFS02-OST0005: Recovery period over after 0:39, of 54 clients 54 recovered and 0 were evicted. May 18 09:32:27 ALPL202 kernel: Lustre: LFS02-OST0005: sending delayed replies to recovered clients May 18 09:32:27 ALPL202 kernel: Lustre: LFS02-OST0005: received MDS connection from 10.3.2.72@o2ib May 18 09:32:27 ALPL202 kernel: Lustre: 8566:0:(filter.c:3123:filter_destroy_precreated()) LFS02-OST0005: deleting orphan objects from 266179450 to 266187588, orphan objids won't be reused any more. May 18 09:32:38 ALPL202 ntpd[6285]: synchronized to 10.3.8.66, stratum 4 May 18 09:34:32 ALPL202 kernel: cm_dreq_handler: no cm_id_priv